-- Logs begin at Mon 2022-07-18 15:49:37 UTC, end at Mon 2022-07-18 17:09:59 UTC. -- Jul 18 16:28:52.814446 dv-cinder-846173-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 16:28:52 2022] *** Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: nodename: dv-cinder-846173-3-iscsi Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: machine: x86_64 Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: clock source: unix Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: pcre jit disabled Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: detected number of CPU cores: 4 Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: current working directory: / Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: detected binary path: /usr/bin/uwsgi-core Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: your processes number limit is 47680 Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: your memory page size is 4096 bytes Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: detected max file descriptor number: 2048 Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: lock engine: pthread robust mutexes Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: thunder lock: enabled Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 16:28:52.832261 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 16:28:52.865209 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: Python main interpreter initialized at 0x5561ddc6c9c0 Jul 18 16:28:52.865209 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: python threads support enabled Jul 18 16:28:52.865209 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: your server socket listen backlog is limited to 100 connections Jul 18 16:28:52.865209 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: your mercy for graceful operations on workers is 90 seconds Jul 18 16:28:52.865571 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 16:28:52.865641 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: *** Operational MODE: preforking *** Jul 18 16:28:52.865907 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: *** uWSGI is running in multiple interpreter mode *** Jul 18 16:28:52.865907 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: spawned uWSGI master process (pid: 80510) Jul 18 16:28:52.866116 dv-cinder-846173-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 16:28:52.867259 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: spawned uWSGI worker 1 (pid: 80522, cores: 1) Jul 18 16:28:52.867259 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: spawned uWSGI worker 2 (pid: 80523, cores: 1) Jul 18 16:28:52.867259 dv-cinder-846173-3-iscsi devstack@keystone.service[80510]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 16:28:58.365648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:28:58.365648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: from cryptography.utils import int_from_bytes Jul 18 16:28:58.366450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:28:58.366450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: from cryptography.utils import int_from_bytes Jul 18 16:28:58.366560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:28:58.366560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: from cryptography.utils import int_from_bytes Jul 18 16:28:58.366970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 16:28:58.366970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: from cryptography.utils import int_from_bytes Jul 18 16:28:58.415517 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 16:28:58.415715 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 16:28:58.415905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] command line args: [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 16:28:58.416002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 16:28:58.416128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 16:28:58.416211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 16:28:58.416329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] command line args: [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 16:28:58.416412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ================================================================================ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 16:28:58.416587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] admin_token = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.416664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 16:28:58.416772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ================================================================================ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 16:28:58.416855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config_dir = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.417039 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.417137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] admin_token = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.417213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] config_source = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.417288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config_dir = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.417390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] debug = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.417473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.417592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.417846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] config_source = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.417846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] debug = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.417970 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.418032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.418132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.418237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.418504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.418682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.418850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.419000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.419117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.419225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.419319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_config_append = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.419422 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.419540 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.419694 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_config_append = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.419841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_dir = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.419978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.420067 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.420188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_dir = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.420268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.420386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.420465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.420582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.420666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.420742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.420860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.420977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.421099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.421180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.421262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.421409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.421484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.421602 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.421682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.421763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.421914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.422507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.422580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.422718 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.422839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.422968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.423048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.423123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.423235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.423322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.423402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.423519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.423596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.423671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.423790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.423954 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.424135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] publish_errors = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.424218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.424370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.424456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] publish_errors = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.424608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.424727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.424799 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.424940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.425043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.425130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.425314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.425494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.425567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] standard_threads = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.425633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.425735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.425811 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] standard_threads = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.425967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.426044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.426139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.426298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.426376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_journal = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.426451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.426526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_json = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.426644 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_journal = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.426728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_stderr = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.426809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_json = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.426884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] use_syslog = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.426958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_stderr = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.427032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.427106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] use_syslog = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.427260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.427347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 16:28:58.427500 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.427577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.427747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.427828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.427991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.428072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.428244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.428326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.428484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.428565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.428729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.428815 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.428993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.429071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433604 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433824 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433894 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.433973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.434046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.434137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.434215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.434292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.434368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.434447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.434518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.434644 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.434719 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.434861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.434939 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.435981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436122 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436556 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436631 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.436978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.437084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.437185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.437279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.437377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.437527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.437660 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.437768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.437870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.437965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.438057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.438173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.438259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.438331 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.438455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.438455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.438630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.438704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.438816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.438896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.438973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.439088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.439170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.439241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.439353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.439427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.439544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.439626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.439702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.439773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.439886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.439960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.440071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.440151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.440223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.440865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.440983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.441103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.441103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.441287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.441363 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.441475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.441549 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.441661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.441734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.441816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.441892 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442758 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442844 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.442914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443109 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443109 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443438 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443780 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.443994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444477 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.444969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.445052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.445122 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.445191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.445191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.445315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.445429 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.445511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.445648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.445727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.445897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.445974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.446161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.446247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.446414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.446492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.446668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.446743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.446912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.447000 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.447155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.447234 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.447396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.447473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.447638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.447726 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.447878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.447961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.448145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.448223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.448389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.448477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.448634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.448717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.448892 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.448971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.449149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.449238 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.449390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.449466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.449632 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.449715 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.449873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.449958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.450107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.450185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.450344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.450421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.450579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.450765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.450848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.451014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.451091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.451268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.451360 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.451508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.451591 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.451754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.451835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.451988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.external = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.452073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.452220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.452297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.452499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.452581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.452738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.452821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.452996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.external = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.453150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.password = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.453254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] auth.token = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.453332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.453495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.453573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.453733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.453821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.453981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.454065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.password = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.454216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.454292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] auth.token = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.454469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.454554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.454711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.454786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.454933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.455012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.455163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.455263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.455416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.455531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.455667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.455802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.455927 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.456062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.456184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.456315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.456439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.456563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.456563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.456810 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.456967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.457088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.457088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.457316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.457316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.457629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.457629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.457865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.458005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.458129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.458252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.458383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.458507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.458635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.458760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.458883 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.459005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.459137 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.459137 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.459369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.459499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.459620 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.459754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.459889 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.460012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.460164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.460239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.460350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.460479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.460612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.460741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.460861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.460998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.461121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.461239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.461378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.461502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.461626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.461755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.461880 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.462010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.462134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.462259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.462379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.462518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.462641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.462772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.462897 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.463027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.463160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.463284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.463408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.463539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.463662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.463792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.463921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.464046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.464169 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.464301 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.464425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.464548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.464671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.464804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.464938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.465071 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.465196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.465319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.465449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.465573 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.465695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.465826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.465946 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.466065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.466181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.466302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.466421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.466588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.466723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.466849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.466973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.467106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.467231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.467362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.467486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.467616 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.467739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.467862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.467992 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.468124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.468264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.468389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.468512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.469222 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.469354 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.469490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.469615 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.469739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.469862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.469986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.470110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.470242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.470368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.470492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.470631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.470761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.470884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.471014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.471139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.471263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.471387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.471510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.471634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.471765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.471889 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.472012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.472136 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.472259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.472387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.472515 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.472639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.472762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.472900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.473021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.473141 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.473271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.473397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.473521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.473645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.473768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.473891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.474036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.474161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.474284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.474407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.474532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.474656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.474788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.474911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.475035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.475158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.475282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.475413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.475539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.475662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.475795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.475919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.476050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.476182 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.476307 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.476430 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.476561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.476685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.476813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.477003 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.477123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.477247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.477397 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.477512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.477642 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.477774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.477899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.478022 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.478146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.478146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.478379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.478510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.478634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.478757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.478881 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.479004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.479126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.479126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.479377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.479500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.479623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.479747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.479870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.480001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.480125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.480248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.480371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.480494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.480617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.480748 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.480873 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.481028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.481150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.481271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.481391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.481518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.481638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.481757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.481878 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.481997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.482116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.482312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.482437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.482559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.482679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.482810 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.482930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.483057 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.483177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.483297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.483417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.483536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.483655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.483655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.483889 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.483889 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.484114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.484194 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.485006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.485142 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.485264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.485388 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.485512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.485635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.485758 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.485890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.486015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.486139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.486261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.486384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.486507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.486638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.486771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.486896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.487020 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.487144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.487267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.487401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.487525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.487648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.487772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.487895 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.488019 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.488151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.488274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.488397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.488530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.488713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.488857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.489021 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.489100 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.489224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.489345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.489484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.489606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.489690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.489762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.489832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.489902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.489971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490680 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.490967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491603 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.491961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492030 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492100 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492457 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492603 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.492956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.493883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.494042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.494127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.494280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.494357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.494530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.494857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.495024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.495103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.495260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.495338 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.495501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.495579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.495739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.495816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.495983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.496061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.496221 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.496552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.496552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.496702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.496773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.496842 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.496932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.497004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.497160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.497240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.497404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.497486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.497681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.497760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.497923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.498002 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.498165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.498244 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.498404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.498550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.498638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.498711 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.498890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.498982 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.499125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.499203 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.499377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.499463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.499619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.499696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.499858 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.499983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.500121 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.500272 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.500346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.500425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.500583 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.500661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.500829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.500951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.501091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.501499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.501634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.501711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.501781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.501851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.501921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.501998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.502069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.502140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.502250 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.502328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.502484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.502570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.502723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.502801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.502966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.503042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.503138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 16:28:58.503225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.503450 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.503690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.503938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.504184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 16:28:58.504500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80523) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 16:28:58.535025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.535259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.535352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.535427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.536156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.536254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.544119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.544274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.572544 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.572722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.572922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.573012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.724971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.732857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.774861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.775096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.778783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.779021 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.840401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.840621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.840845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.840952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.841091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.841173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.841287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.841370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.841501 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.841586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.841739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.841961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.867225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.867417 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.867508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.867598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.867704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.867876 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.867968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.868098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.868181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.868319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.868419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.868534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.868626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.868751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.880539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.881169 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.893811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.894046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.894313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.894480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.894568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.894718 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.894824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.894950 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.895044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.895175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.895258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.895393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.895610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.895828 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80523) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 16:28:58.912280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.912483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:28:58.912623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.912836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.912945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:28:58.913061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:28:58.913205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.913350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.913447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 16:28:58.913610 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.913692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.913868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.913949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.914118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.914208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.914405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.925401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.925624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.925821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.925905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.926102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.926187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.926375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.926667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.934376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.934941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.938259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.938534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.938811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.938971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.939111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.939261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.939404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.939489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.939714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.939801 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.939978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.940061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.940245 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.940328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.940509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.940592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.940780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.940861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.941074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.941168 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.941354 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.941625 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.968722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.969335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.970405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.970695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.970999 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.971286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.976016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.976257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.976535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.976676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.976943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.977201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.983825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.984100 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.984582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.984869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.988796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.989090 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.989370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.989559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:58.989702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.989852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.989978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.990138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.990279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.990405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.990528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.990652 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.990775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.990977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.991188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:58.991460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.010243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.010596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.011093 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.011375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.015199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.016606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.019563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.019849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.020391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.020669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.024855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.025767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.028522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.029481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.034300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.034300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.034300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.034300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.034578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.034647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.034708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.034769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.034837 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.034837 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.034837 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.034995 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.052864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.053259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.053527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.053796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.054192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.054576 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.054847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.055121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.061717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.063669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.063995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.064267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.065427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.065707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.067344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.067635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.067923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.068208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.068490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.068783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.069094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.069391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.069687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.069981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.082614 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.082891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.083167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.083441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.083716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.084002 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.084126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.084274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.084408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.084692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.085002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.085289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.085567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.085846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.103667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.105382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.106935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.108684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.110457 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.112057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.112156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.113713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.113897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.113993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.114264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.115376 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.115658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.115937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.121977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.123582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.123678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.125866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.127495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.127777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.128053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.128328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.130384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.130681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.130971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.131250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.137602 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.137885 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.138160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.141144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.141444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.141735 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.146610 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.146885 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.147162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.147431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.147706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.147986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.148253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.148523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.148789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.149085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.149360 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.149630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.149710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80523) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 16:28:59.149890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.149971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.150143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.150226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.150469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.150745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.151035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.151315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.151582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.151853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.152124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.152417 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.152683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.152975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.153242 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80523) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 16:28:59.188485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5561ddc6c9c0 pid: 80522 (default app) Jul 18 16:28:59.191774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5561ddc6c9c0 pid: 80523 (default app) Jul 18 16:28:59.191873 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7e6c-3767-44e6-a5e6-7ae8fca0b9c9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:28:59.192169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7e6c-3767-44e6-a5e6-7ae8fca0b9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:28:59.192449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7e6c-3767-44e6-a5e6-7ae8fca0b9c9 None None] PATH_INFO: `/v3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:28:59.196619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1/1] 10.222.0.88 () {50 vars in 791 bytes} [Mon Jul 18 16:28:59 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:07.760138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a15e2-abe5-4b54-a2fb-590d6c8d6b86 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:07.772679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a15e2-abe5-4b54-a2fb-590d6c8d6b86 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:07.772679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a15e2-abe5-4b54-a2fb-590d6c8d6b86 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:07.772679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1/2] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:07.772966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00256e-bedd-4ee4-87a6-9502d2d5e538 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:07.772966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00256e-bedd-4ee4-87a6-9502d2d5e538 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:07.772966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00256e-bedd-4ee4-87a6-9502d2d5e538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:07.844284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: INFO dbcounter [None req-5d00256e-bedd-4ee4-87a6-9502d2d5e538 None None] Registered counter for database keystone Jul 18 16:29:07.867792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG oslo_db.sqlalchemy.engines [None req-5d00256e-bedd-4ee4-87a6-9502d2d5e538 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80522) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 16:29:07.875481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writer thread running {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 16:29:08.062813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG passlib.handlers.bcrypt [None req-5d00256e-bedd-4ee4-87a6-9502d2d5e538 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80522) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 16:29:08.063155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG passlib.handlers.bcrypt [None req-5d00256e-bedd-4ee4-87a6-9502d2d5e538 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80522) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 16:29:08.121023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5d00256e-bedd-4ee4-87a6-9502d2d5e538 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:08.122651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5d00256e-bedd-4ee4-87a6-9502d2d5e538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:08.258926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 2/3] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 16:29:08.444219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a15e2-abe5-4b54-a2fb-590d6c8d6b86 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:08.515933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: INFO dbcounter [None req-829a15e2-abe5-4b54-a2fb-590d6c8d6b86 None None] Registered counter for database keystone Jul 18 16:29:08.536016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG oslo_db.sqlalchemy.engines [None req-829a15e2-abe5-4b54-a2fb-590d6c8d6b86 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80523) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 16:29:08.542321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writer thread running {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 16:29:08.661355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a15e2-abe5-4b54-a2fb-590d6c8d6b86 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:08.665624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f3dec-4d2f-4c11-becf-fd4b9350609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:08.666289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f3dec-4d2f-4c11-becf-fd4b9350609c admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:08.666581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f3dec-4d2f-4c11-becf-fd4b9350609c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:08.666852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f3dec-4d2f-4c11-becf-fd4b9350609c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:08.675231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f3dec-4d2f-4c11-becf-fd4b9350609c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:08.727707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a31f3dec-4d2f-4c11-becf-fd4b9350609c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 16:29:08.727979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f3dec-4d2f-4c11-becf-fd4b9350609c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:08.738418 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 2/4] 10.222.0.88 () {58 vars in 1324 bytes} [Mon Jul 18 16:29:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:29:08.744735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00256e-bedd-4ee4-87a6-9502d2d5e538 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:08.759743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00256e-bedd-4ee4-87a6-9502d2d5e538 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:08.764175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9eb10-0dfb-42bf-89e4-3bba4a0df9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:08.765928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9eb10-0dfb-42bf-89e4-3bba4a0df9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:08.766221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9eb10-0dfb-42bf-89e4-3bba4a0df9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:08.766502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9eb10-0dfb-42bf-89e4-3bba4a0df9a7 admin admin] PATH_INFO: `/v3/projects/fb46b30ed74742b9a95d61944ef39381` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:08.768025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9eb10-0dfb-42bf-89e4-3bba4a0df9a7 admin admin] RBAC: Authorizing `identity:get_project(project_id=fb46b30ed74742b9a95d61944ef39381)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:08.821554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7fb9eb10-0dfb-42bf-89e4-3bba4a0df9a7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 16:29:08.822279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9eb10-0dfb-42bf-89e4-3bba4a0df9a7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:08.826619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 3/5] 10.222.0.88 () {56 vars in 1207 bytes} [Mon Jul 18 16:29:08 2022] GET /identity/v3/projects/fb46b30ed74742b9a95d61944ef39381 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:10.821320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7d0e9-157d-4479-b735-f48b7c078522 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:10.821838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7d0e9-157d-4479-b735-f48b7c078522 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:10.821952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7d0e9-157d-4479-b735-f48b7c078522 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:10.826067 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 3/6] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:29:10.831669 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4291c9d-e6e8-4a01-bf98-944c3dfa6eeb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:10.832100 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4291c9d-e6e8-4a01-bf98-944c3dfa6eeb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:10.832446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4291c9d-e6e8-4a01-bf98-944c3dfa6eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:10.851650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f4291c9d-e6e8-4a01-bf98-944c3dfa6eeb None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:10.853231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f4291c9d-e6e8-4a01-bf98-944c3dfa6eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:10.861734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 4/7] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 16:29:10.884800 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7d0e9-157d-4479-b735-f48b7c078522 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:10.897086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7d0e9-157d-4479-b735-f48b7c078522 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:10.901242 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e51102-afcf-4b62-bbaa-59eaf89d6705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:10.901870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e51102-afcf-4b62-bbaa-59eaf89d6705 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:10.902152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e51102-afcf-4b62-bbaa-59eaf89d6705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:10.902425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e51102-afcf-4b62-bbaa-59eaf89d6705 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:10.910709 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e51102-afcf-4b62-bbaa-59eaf89d6705 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:10.912473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e51102-afcf-4b62-bbaa-59eaf89d6705 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:10.921308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 4/8] 10.222.0.88 () {58 vars in 1324 bytes} [Mon Jul 18 16:29:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:29:10.927519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4291c9d-e6e8-4a01-bf98-944c3dfa6eeb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:10.936999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4291c9d-e6e8-4a01-bf98-944c3dfa6eeb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:10.941501 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf8d73-008b-4e01-8590-65ceca2c2b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:10.942374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf8d73-008b-4e01-8590-65ceca2c2b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:10.942663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf8d73-008b-4e01-8590-65ceca2c2b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:10.942943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf8d73-008b-4e01-8590-65ceca2c2b43 admin admin] PATH_INFO: `/v3/users/264c9bf15b9849a18b942d5d77559b9b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:10.944344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf8d73-008b-4e01-8590-65ceca2c2b43 admin admin] RBAC: Authorizing `identity:get_user(user_id=264c9bf15b9849a18b942d5d77559b9b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:10.946230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf8d73-008b-4e01-8590-65ceca2c2b43 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:10.947820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 5/9] 10.222.0.88 () {56 vars in 1198 bytes} [Mon Jul 18 16:29:10 2022] GET /identity/v3/users/264c9bf15b9849a18b942d5d77559b9b => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:13.028132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e658d-d549-4935-baef-499ae04169c6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.028689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e658d-d549-4935-baef-499ae04169c6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.028689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e658d-d549-4935-baef-499ae04169c6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.030088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 5/10] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:29:13.035985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fdc3b-fff4-413a-98d1-b732203fd136 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.036411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fdc3b-fff4-413a-98d1-b732203fd136 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.036640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fdc3b-fff4-413a-98d1-b732203fd136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.037620 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9cca1-3b6e-485a-8007-0fd4891f72bb None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.037914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9cca1-3b6e-485a-8007-0fd4891f72bb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.038204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9cca1-3b6e-485a-8007-0fd4891f72bb None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.039496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 6/11] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:13.044451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9206a98-a502-4018-a90f-9634a4102181 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.044740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9206a98-a502-4018-a90f-9634a4102181 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.045055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9206a98-a502-4018-a90f-9634a4102181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.054420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-298fdc3b-fff4-413a-98d1-b732203fd136 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:13.055627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-298fdc3b-fff4-413a-98d1-b732203fd136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:13.085362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG passlib.handlers.bcrypt [None req-e9206a98-a502-4018-a90f-9634a4102181 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80523) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 16:29:13.085779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG passlib.handlers.bcrypt [None req-e9206a98-a502-4018-a90f-9634a4102181 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80523) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 16:29:13.131966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e9206a98-a502-4018-a90f-9634a4102181 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:13.133610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e9206a98-a502-4018-a90f-9634a4102181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:13.138053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 6/12] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:13.161625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fdc3b-fff4-413a-98d1-b732203fd136 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:13.190701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:13.238548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 7/13] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:13.261000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9206a98-a502-4018-a90f-9634a4102181 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:13.274050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fdc3b-fff4-413a-98d1-b732203fd136 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:13.309016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc125f0-a1e9-48b8-b581-6adcb6d43a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:13.309817 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc125f0-a1e9-48b8-b581-6adcb6d43a4a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.310108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc125f0-a1e9-48b8-b581-6adcb6d43a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.310384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc125f0-a1e9-48b8-b581-6adcb6d43a4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.322166 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc125f0-a1e9-48b8-b581-6adcb6d43a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-acc125f0-a1e9-48b8-b581-6adcb6d43a4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:13.333454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:13.334414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:13.335322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 7/14] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:29:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:13.340521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc125f0-a1e9-48b8-b581-6adcb6d43a4a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:13.363853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9206a98-a502-4018-a90f-9634a4102181 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:13.366199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:13.403512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d4934-b444-4238-b740-c228ac4d2f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:13.404419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d4934-b444-4238-b740-c228ac4d2f95 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.404712 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d4934-b444-4238-b740-c228ac4d2f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.405018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d4934-b444-4238-b740-c228ac4d2f95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.427978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:13.445932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc125f0-a1e9-48b8-b581-6adcb6d43a4a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:13.475762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d4934-b444-4238-b740-c228ac4d2f95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:13.477345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d4934-b444-4238-b740-c228ac4d2f95 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:13.481717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4287296-33f9-4570-b2a1-d7e65c1cbd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:13.482392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4287296-33f9-4570-b2a1-d7e65c1cbd09 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.482680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4287296-33f9-4570-b2a1-d7e65c1cbd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.482962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4287296-33f9-4570-b2a1-d7e65c1cbd09 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.483755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4287296-33f9-4570-b2a1-d7e65c1cbd09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:13.484615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4287296-33f9-4570-b2a1-d7e65c1cbd09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:13.486057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4287296-33f9-4570-b2a1-d7e65c1cbd09 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:13.498540 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 8/15] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:13.504101 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4287296-33f9-4570-b2a1-d7e65c1cbd09 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:13.576572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 8/16] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:29:13.582716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d4934-b444-4238-b740-c228ac4d2f95 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:13.585641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:13.609040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4287296-33f9-4570-b2a1-d7e65c1cbd09 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:13.643976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515601f2-374c-4266-b68f-7198502494cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:13.644858 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515601f2-374c-4266-b68f-7198502494cc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.645177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515601f2-374c-4266-b68f-7198502494cc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.645461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515601f2-374c-4266-b68f-7198502494cc None admin] PATH_INFO: `/v3/users/264c9bf15b9849a18b942d5d77559b9b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.646691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515601f2-374c-4266-b68f-7198502494cc None admin] RBAC: Authorizing `identity:get_user(user_id=264c9bf15b9849a18b942d5d77559b9b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:13.648160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515601f2-374c-4266-b68f-7198502494cc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:13.649843 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 9/17] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:13 2022] GET /identity/v3/users/264c9bf15b9849a18b942d5d77559b9b => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:13.655633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515601f2-374c-4266-b68f-7198502494cc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:13.671585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:13.684442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d4934-b444-4238-b740-c228ac4d2f95 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:13.720546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a7640-7fe9-4e0d-ae4c-309f87d93e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:13.721277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a7640-7fe9-4e0d-ae4c-309f87d93e80 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.721566 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a7640-7fe9-4e0d-ae4c-309f87d93e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.721845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a7640-7fe9-4e0d-ae4c-309f87d93e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.740759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a7640-7fe9-4e0d-ae4c-309f87d93e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:13.742275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a7640-7fe9-4e0d-ae4c-309f87d93e80 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:13.758314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515601f2-374c-4266-b68f-7198502494cc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-0c5a7640-7fe9-4e0d-ae4c-309f87d93e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:29:13.766173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:13.767137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:13.767912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:29:13.767912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:29:13.767912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:29:13.767912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:13.767912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 9/18] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:13.773418 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a7640-7fe9-4e0d-ae4c-309f87d93e80 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:13.794838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5de482-cd6b-4160-8d51-260a71597ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:13.795633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5de482-cd6b-4160-8d51-260a71597ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.795919 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5de482-cd6b-4160-8d51-260a71597ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.796202 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5de482-cd6b-4160-8d51-260a71597ca6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.797991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5de482-cd6b-4160-8d51-260a71597ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:13.799439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5de482-cd6b-4160-8d51-260a71597ca6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:13.801251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 10/19] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:13.806666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5de482-cd6b-4160-8d51-260a71597ca6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:13.834157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:13.882095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a7640-7fe9-4e0d-ae4c-309f87d93e80 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:13.890480 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:13.910951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5de482-cd6b-4160-8d51-260a71597ca6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:13.916821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f34fb3-de3b-49d2-b881-22305c1b369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:13.917505 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f34fb3-de3b-49d2-b881-22305c1b369e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.917787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f34fb3-de3b-49d2-b881-22305c1b369e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.918060 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f34fb3-de3b-49d2-b881-22305c1b369e None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.918781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f34fb3-de3b-49d2-b881-22305c1b369e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:13.919084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f34fb3-de3b-49d2-b881-22305c1b369e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:13.920481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f34fb3-de3b-49d2-b881-22305c1b369e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:13.939862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 10/20] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:13.949797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56206a-9e1a-4605-8f18-d1d30ccc9367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:13.950490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56206a-9e1a-4605-8f18-d1d30ccc9367 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:13.950803 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56206a-9e1a-4605-8f18-d1d30ccc9367 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:13.951069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56206a-9e1a-4605-8f18-d1d30ccc9367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:13.951948 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56206a-9e1a-4605-8f18-d1d30ccc9367 None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.domain.id=default, user.id=264c9bf15b9849a18b942d5d77559b9b) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:13.952261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56206a-9e1a-4605-8f18-d1d30ccc9367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:13.953874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56206a-9e1a-4605-8f18-d1d30ccc9367 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:13.962470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 11/21] 10.222.0.88 () {56 vars in 1322 bytes} [Mon Jul 18 16:29:13 2022] GET /identity/v3/role_assignments?user.id=264c9bf15b9849a18b942d5d77559b9b&scope.domain.id=default&role.id=81457691f3a2456fa33358cb6a922d05 => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:16.252430 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b8734-47c0-4aeb-81f9-4827fdcaf52c None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:16.253475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b8734-47c0-4aeb-81f9-4827fdcaf52c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:16.253966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b8734-47c0-4aeb-81f9-4827fdcaf52c None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:16.255597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 11/22] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:16.263021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b931cc1-c9c7-47ff-affe-38a7a738a8b5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:16.263934 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b931cc1-c9c7-47ff-affe-38a7a738a8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:16.264425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b931cc1-c9c7-47ff-affe-38a7a738a8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:16.297955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9b931cc1-c9c7-47ff-affe-38a7a738a8b5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:16.299194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9b931cc1-c9c7-47ff-affe-38a7a738a8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:16.366409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dab20-a7c2-4df6-a43f-d81d7e8d29f3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:16.367031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dab20-a7c2-4df6-a43f-d81d7e8d29f3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:16.367532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dab20-a7c2-4df6-a43f-d81d7e8d29f3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:16.369144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 12/23] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:29:16.382259 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711e0ac-ec2e-45ef-9b8b-bbb7c2509c7f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:16.382840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711e0ac-ec2e-45ef-9b8b-bbb7c2509c7f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:16.383310 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711e0ac-ec2e-45ef-9b8b-bbb7c2509c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:16.422290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 12/24] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:16.423945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49976da3-2bef-4ae2-a6cf-04a1bff21bf4 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:16.424473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49976da3-2bef-4ae2-a6cf-04a1bff21bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:16.424984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49976da3-2bef-4ae2-a6cf-04a1bff21bf4 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:16.426454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 13/25] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:16.432978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5711e0ac-ec2e-45ef-9b8b-bbb7c2509c7f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:16.432978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5711e0ac-ec2e-45ef-9b8b-bbb7c2509c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:16.438869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a3548-8ae1-4425-821b-4031e4848163 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:16.439422 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a3548-8ae1-4425-821b-4031e4848163 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:16.439909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a3548-8ae1-4425-821b-4031e4848163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:16.469644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-527a3548-8ae1-4425-821b-4031e4848163 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:16.471423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-527a3548-8ae1-4425-821b-4031e4848163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:16.570137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 13/26] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 18 16:29:16.570840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711e0ac-ec2e-45ef-9b8b-bbb7c2509c7f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:16.606478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 14/27] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:16.607340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a3548-8ae1-4425-821b-4031e4848163 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:16.628135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:16.652119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:16.749574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711e0ac-ec2e-45ef-9b8b-bbb7c2509c7f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:16.814807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a3548-8ae1-4425-821b-4031e4848163 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:16.828834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a00b1-5429-4e49-8c8c-9ea5fd4d7260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:16.829591 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a00b1-5429-4e49-8c8c-9ea5fd4d7260 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:16.829873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a00b1-5429-4e49-8c8c-9ea5fd4d7260 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:16.830283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a00b1-5429-4e49-8c8c-9ea5fd4d7260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:16.866025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a00b1-5429-4e49-8c8c-9ea5fd4d7260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:16.867554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a00b1-5429-4e49-8c8c-9ea5fd4d7260 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:16.887527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47cbe4-100d-4544-832f-9ae827b9e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:16.888355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47cbe4-100d-4544-832f-9ae827b9e45c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:16.888646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47cbe4-100d-4544-832f-9ae827b9e45c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:16.888961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47cbe4-100d-4544-832f-9ae827b9e45c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bc7a00b1-5429-4e49-8c8c-9ea5fd4d7260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:29:16.910765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:16.912864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:16.913698 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:29:16.913698 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:29:16.913698 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:29:16.913698 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:16.914922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 14/28] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:16.915779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a00b1-5429-4e49-8c8c-9ea5fd4d7260 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:16.919388 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47cbe4-100d-4544-832f-9ae827b9e45c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-da47cbe4-100d-4544-832f-9ae827b9e45c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:16.928484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:16.929657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:16.930560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:16.933149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 15/29] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:29:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:16.934345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47cbe4-100d-4544-832f-9ae827b9e45c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:17.044378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:17.081256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:17.085926 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a00b1-5429-4e49-8c8c-9ea5fd4d7260 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:17.090119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47cbe4-100d-4544-832f-9ae827b9e45c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:17.123320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7d3e3-19cb-4a62-a43f-ebcb4082a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:17.124948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7d3e3-19cb-4a62-a43f-ebcb4082a8de None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:17.125234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7d3e3-19cb-4a62-a43f-ebcb4082a8de None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:17.125670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7d3e3-19cb-4a62-a43f-ebcb4082a8de None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:17.126586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7d3e3-19cb-4a62-a43f-ebcb4082a8de None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:17.128166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7d3e3-19cb-4a62-a43f-ebcb4082a8de None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:17.129050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f2c3a-655a-4c57-b80c-358ccfb4e22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:17.129952 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2c3a-655a-4c57-b80c-358ccfb4e22a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:17.130444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2c3a-655a-4c57-b80c-358ccfb4e22a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:17.130994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f2c3a-655a-4c57-b80c-358ccfb4e22a None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:17.131933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f2c3a-655a-4c57-b80c-358ccfb4e22a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:17.132530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f2c3a-655a-4c57-b80c-358ccfb4e22a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:17.134835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f2c3a-655a-4c57-b80c-358ccfb4e22a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:17.155750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 16/30] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:17.157033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f2c3a-655a-4c57-b80c-358ccfb4e22a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:17.282025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f2c3a-655a-4c57-b80c-358ccfb4e22a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:17.310423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:17.322857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f69330-1394-470e-ab3a-22d2fff3a88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:17.323787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69330-1394-470e-ab3a-22d2fff3a88f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:17.324236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69330-1394-470e-ab3a-22d2fff3a88f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:17.326369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f69330-1394-470e-ab3a-22d2fff3a88f None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:17.326369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f69330-1394-470e-ab3a-22d2fff3a88f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:17.326369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f69330-1394-470e-ab3a-22d2fff3a88f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:17.327691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f69330-1394-470e-ab3a-22d2fff3a88f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:17.337425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 17/31] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:17.339362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f346e-be4a-4721-9153-597fa787cae9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:17.339836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f346e-be4a-4721-9153-597fa787cae9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:17.340275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f346e-be4a-4721-9153-597fa787cae9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:17.342195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 18/32] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:17.343242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f346e-be4a-4721-9153-597fa787cae9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:17.368175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 15/33] 10.222.0.88 () {60 vars in 1131 bytes} [Mon Jul 18 16:29:16 2022] POST /identity/v3/roles => generated 219 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:29:17.370113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7d3e3-19cb-4a62-a43f-ebcb4082a8de None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:17.484223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f346e-be4a-4721-9153-597fa787cae9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:17.507718 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7d3e3-19cb-4a62-a43f-ebcb4082a8de None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:17.540666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21bb66-8665-4499-a7f5-698c07e3a658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:17.541434 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21bb66-8665-4499-a7f5-698c07e3a658 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:17.541755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21bb66-8665-4499-a7f5-698c07e3a658 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:17.542117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21bb66-8665-4499-a7f5-698c07e3a658 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:17.542910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21bb66-8665-4499-a7f5-698c07e3a658 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:17.544422 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21bb66-8665-4499-a7f5-698c07e3a658 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:17.578943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accb04e-a1d8-4ebb-8520-47d786291c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:17.581720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accb04e-a1d8-4ebb-8520-47d786291c61 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:17.582015 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accb04e-a1d8-4ebb-8520-47d786291c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:17.582298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accb04e-a1d8-4ebb-8520-47d786291c61 None admin] PATH_INFO: `/v3/users/264c9bf15b9849a18b942d5d77559b9b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:17.587293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accb04e-a1d8-4ebb-8520-47d786291c61 None admin] RBAC: Authorizing `identity:get_user(user_id=264c9bf15b9849a18b942d5d77559b9b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:17.589462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accb04e-a1d8-4ebb-8520-47d786291c61 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:17.591614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 16/34] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:17 2022] GET /identity/v3/users/264c9bf15b9849a18b942d5d77559b9b => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:17.597938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5d308-bb65-4455-8776-59ba94c3402c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:17.598222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5d308-bb65-4455-8776-59ba94c3402c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:17.598494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5d308-bb65-4455-8776-59ba94c3402c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:17.618745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cfd5d308-bb65-4455-8776-59ba94c3402c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:17.621102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cfd5d308-bb65-4455-8776-59ba94c3402c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:17.644326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:17.707633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 17/35] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:17.708498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5d308-bb65-4455-8776-59ba94c3402c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:17.835461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5d308-bb65-4455-8776-59ba94c3402c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:17.878883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46f712-a80a-48ee-9f2f-3713adc1946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:17.879688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46f712-a80a-48ee-9f2f-3713adc1946d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:17.879971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46f712-a80a-48ee-9f2f-3713adc1946d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:17.880247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46f712-a80a-48ee-9f2f-3713adc1946d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:17.881590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46f712-a80a-48ee-9f2f-3713adc1946d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:17.883335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46f712-a80a-48ee-9f2f-3713adc1946d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:17.885196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 18/36] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:17.885967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46f712-a80a-48ee-9f2f-3713adc1946d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:17.913934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:17.945514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 19/37] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:29:17 2022] POST /identity/v3/projects => generated 301 bytes in 603 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:29:17.947091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd427-a304-4fd7-bb95-c26887bfcc49 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:17.947372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd427-a304-4fd7-bb95-c26887bfcc49 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:17.947749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd427-a304-4fd7-bb95-c26887bfcc49 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:17.949126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 20/38] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:17.950210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1d8d7-249a-482f-a2be-fe5bf57d5b10 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:17.950502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1d8d7-249a-482f-a2be-fe5bf57d5b10 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:17.950791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1d8d7-249a-482f-a2be-fe5bf57d5b10 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:17.956146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 21/39] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:17.956146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1d8d7-249a-482f-a2be-fe5bf57d5b10 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:18.006471 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46f712-a80a-48ee-9f2f-3713adc1946d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:18.009219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:18.053677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a58f24-f5d6-423b-9f3a-81e5a45a7d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:18.054523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a58f24-f5d6-423b-9f3a-81e5a45a7d71 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:18.054813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a58f24-f5d6-423b-9f3a-81e5a45a7d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:18.055098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a58f24-f5d6-423b-9f3a-81e5a45a7d71 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:18.055881 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a58f24-f5d6-423b-9f3a-81e5a45a7d71 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:18.057353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a58f24-f5d6-423b-9f3a-81e5a45a7d71 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:18.085544 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1d8d7-249a-482f-a2be-fe5bf57d5b10 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:18.126687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e3b96-ec67-40b0-bc07-ca5633670bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:18.128064 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e3b96-ec67-40b0-bc07-ca5633670bf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:18.128722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e3b96-ec67-40b0-bc07-ca5633670bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:18.129277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e3b96-ec67-40b0-bc07-ca5633670bf8 None admin] PATH_INFO: `/v3/domains/default/users/264c9bf15b9849a18b942d5d77559b9b/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:18.132091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e3b96-ec67-40b0-bc07-ca5633670bf8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=264c9bf15b9849a18b942d5d77559b9b, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:18.133495 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 19/40] 10.222.0.88 () {60 vars in 1131 bytes} [Mon Jul 18 16:29:17 2022] POST /identity/v3/roles => generated 225 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:29:18.134374 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629f813-9c50-4cb6-a261-e983d7ba24e8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:18.134676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629f813-9c50-4cb6-a261-e983d7ba24e8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:18.134965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629f813-9c50-4cb6-a261-e983d7ba24e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:18.135784 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e3b96-ec67-40b0-bc07-ca5633670bf8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:18.161333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4629f813-9c50-4cb6-a261-e983d7ba24e8 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:18.162743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4629f813-9c50-4cb6-a261-e983d7ba24e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:18.245237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 22/41] 10.222.0.88 () {58 vars in 1361 bytes} [Mon Jul 18 16:29:17 2022] PUT /identity/v3/domains/default/users/264c9bf15b9849a18b942d5d77559b9b/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:29:18.252462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 20/42] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:18.252695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ce276-65ae-4aff-bf7e-b0d3a436abc1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:18.252695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ce276-65ae-4aff-bf7e-b0d3a436abc1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:18.252695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ce276-65ae-4aff-bf7e-b0d3a436abc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:18.268409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ad8ce276-65ae-4aff-bf7e-b0d3a436abc1 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:18.270140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ad8ce276-65ae-4aff-bf7e-b0d3a436abc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:18.290073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629f813-9c50-4cb6-a261-e983d7ba24e8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:18.309174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:18.368790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 23/43] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:18.397141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ce276-65ae-4aff-bf7e-b0d3a436abc1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:18.426336 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629f813-9c50-4cb6-a261-e983d7ba24e8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:18.438393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:18.476188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbcd8b-7b04-420a-ab6b-09ccbb2774ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:18.477667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbcd8b-7b04-420a-ab6b-09ccbb2774ae None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:18.478397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbcd8b-7b04-420a-ab6b-09ccbb2774ae None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:18.478850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbcd8b-7b04-420a-ab6b-09ccbb2774ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:18.480534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbcd8b-7b04-420a-ab6b-09ccbb2774ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:18.482486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbcd8b-7b04-420a-ab6b-09ccbb2774ae None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:18.486506 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 21/44] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:18.491390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbcd8b-7b04-420a-ab6b-09ccbb2774ae None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:18.547480 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ce276-65ae-4aff-bf7e-b0d3a436abc1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:18.582542 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:18.587121 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668ce09-49be-45e1-a480-a8fcdfa37e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:18.587807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668ce09-49be-45e1-a480-a8fcdfa37e9c None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:18.588093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668ce09-49be-45e1-a480-a8fcdfa37e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:18.588367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668ce09-49be-45e1-a480-a8fcdfa37e9c None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:18.589162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668ce09-49be-45e1-a480-a8fcdfa37e9c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:18.590967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668ce09-49be-45e1-a480-a8fcdfa37e9c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:18.604935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbcd8b-7b04-420a-ab6b-09ccbb2774ae None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:18.643635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6373b3-abef-4c4c-9ed9-b17eafb24ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:18.644308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6373b3-abef-4c4c-9ed9-b17eafb24ec8 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:18.644587 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6373b3-abef-4c4c-9ed9-b17eafb24ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:18.644859 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6373b3-abef-4c4c-9ed9-b17eafb24ec8 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:18.645603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6373b3-abef-4c4c-9ed9-b17eafb24ec8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:18.647059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6373b3-abef-4c4c-9ed9-b17eafb24ec8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:18.665024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 24/45] 10.222.0.88 () {60 vars in 1131 bytes} [Mon Jul 18 16:29:18 2022] POST /identity/v3/roles => generated 223 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:29:18.738381 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 22/46] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:29:18 2022] POST /identity/v3/projects => generated 312 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:29:20.383908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42beb07-accf-4300-86cc-0e5e90c2fdd8 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:20.384425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42beb07-accf-4300-86cc-0e5e90c2fdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:20.384529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42beb07-accf-4300-86cc-0e5e90c2fdd8 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:20.385927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 25/47] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:20.392214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791f15c-ad6a-41ba-afe3-4fa5bd0ad312 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:20.392670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791f15c-ad6a-41ba-afe3-4fa5bd0ad312 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:20.392670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791f15c-ad6a-41ba-afe3-4fa5bd0ad312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:20.418941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0791f15c-ad6a-41ba-afe3-4fa5bd0ad312 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:20.420168 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0791f15c-ad6a-41ba-afe3-4fa5bd0ad312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:20.489263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 23/48] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:20.512390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42beb07-accf-4300-86cc-0e5e90c2fdd8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:20.604351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:20.617083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42beb07-accf-4300-86cc-0e5e90c2fdd8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:20.651809 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808854f-bd1d-4afd-982a-0e8a69dd9b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:20.652548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808854f-bd1d-4afd-982a-0e8a69dd9b63 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:20.652837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808854f-bd1d-4afd-982a-0e8a69dd9b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:20.653147 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808854f-bd1d-4afd-982a-0e8a69dd9b63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:20.664748 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808854f-bd1d-4afd-982a-0e8a69dd9b63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1808854f-bd1d-4afd-982a-0e8a69dd9b63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:20.673269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:20.674105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:20.674863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 26/49] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:29:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:20.680621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791f15c-ad6a-41ba-afe3-4fa5bd0ad312 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:20.746885 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:20.786779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791f15c-ad6a-41ba-afe3-4fa5bd0ad312 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:20.822130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb6851-a9a3-46da-838d-b62930d0bc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:20.822801 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb6851-a9a3-46da-838d-b62930d0bc4a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:20.823085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb6851-a9a3-46da-838d-b62930d0bc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:20.823361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb6851-a9a3-46da-838d-b62930d0bc4a None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:20.824129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb6851-a9a3-46da-838d-b62930d0bc4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:20.824429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb6851-a9a3-46da-838d-b62930d0bc4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:20.825873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb6851-a9a3-46da-838d-b62930d0bc4a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:20.837830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 24/50] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:20.844047 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808854f-bd1d-4afd-982a-0e8a69dd9b63 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:20.901534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed49cd7-86f4-4183-b2be-742882173bd4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:20.901906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed49cd7-86f4-4183-b2be-742882173bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:20.902084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed49cd7-86f4-4183-b2be-742882173bd4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:20.904032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 25/51] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:29:20.909523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee26ec-1d1a-4c97-9ea6-736392df44d4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:20.909816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee26ec-1d1a-4c97-9ea6-736392df44d4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:20.910101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee26ec-1d1a-4c97-9ea6-736392df44d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:20.928171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c3ee26ec-1d1a-4c97-9ea6-736392df44d4 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:20.930131 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c3ee26ec-1d1a-4c97-9ea6-736392df44d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:20.950690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808854f-bd1d-4afd-982a-0e8a69dd9b63 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:20.995496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a59a04-2dc4-4658-940d-f7108986b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:20.996386 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a59a04-2dc4-4658-940d-f7108986b911 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:20.996675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a59a04-2dc4-4658-940d-f7108986b911 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:20.997033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a59a04-2dc4-4658-940d-f7108986b911 None admin] PATH_INFO: `/v3/users/264c9bf15b9849a18b942d5d77559b9b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:20.998259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a59a04-2dc4-4658-940d-f7108986b911 None admin] RBAC: Authorizing `identity:get_user(user_id=264c9bf15b9849a18b942d5d77559b9b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:20.999757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a59a04-2dc4-4658-940d-f7108986b911 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:21.001502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 27/52] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:20 2022] GET /identity/v3/users/264c9bf15b9849a18b942d5d77559b9b => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:21.006350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 26/53] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:21.007118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a59a04-2dc4-4658-940d-f7108986b911 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:21.028751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:21.029599 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee26ec-1d1a-4c97-9ea6-736392df44d4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:21.129668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a59a04-2dc4-4658-940d-f7108986b911 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:21.167998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240459b-f334-4778-85b4-6ce9a2e2b315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:21.168724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240459b-f334-4778-85b4-6ce9a2e2b315 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:21.169052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240459b-f334-4778-85b4-6ce9a2e2b315 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:21.169334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240459b-f334-4778-85b4-6ce9a2e2b315 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:21.170579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240459b-f334-4778-85b4-6ce9a2e2b315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:21.172115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240459b-f334-4778-85b4-6ce9a2e2b315 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:21.173822 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 28/54] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:21.179700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240459b-f334-4778-85b4-6ce9a2e2b315 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:21.199228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:21.244725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee26ec-1d1a-4c97-9ea6-736392df44d4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:21.269680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:21.282237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923392c5-ba4b-44b5-b96c-4285531ad744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:21.283044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923392c5-ba4b-44b5-b96c-4285531ad744 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:21.283324 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923392c5-ba4b-44b5-b96c-4285531ad744 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:21.283598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923392c5-ba4b-44b5-b96c-4285531ad744 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:21.284802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923392c5-ba4b-44b5-b96c-4285531ad744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:21.286254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240459b-f334-4778-85b4-6ce9a2e2b315 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:21.286361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923392c5-ba4b-44b5-b96c-4285531ad744 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:21.287913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 27/55] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:21.293452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923392c5-ba4b-44b5-b96c-4285531ad744 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:21.320722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632007c4-c580-4c8f-a01c-b15df2fe1338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:21.321412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632007c4-c580-4c8f-a01c-b15df2fe1338 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:21.321702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632007c4-c580-4c8f-a01c-b15df2fe1338 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:21.321983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632007c4-c580-4c8f-a01c-b15df2fe1338 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:21.322827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632007c4-c580-4c8f-a01c-b15df2fe1338 None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.domain.id=default, user.id=264c9bf15b9849a18b942d5d77559b9b) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:21.323128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632007c4-c580-4c8f-a01c-b15df2fe1338 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:21.324591 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632007c4-c580-4c8f-a01c-b15df2fe1338 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:21.330991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 29/56] 10.222.0.88 () {56 vars in 1322 bytes} [Mon Jul 18 16:29:21 2022] GET /identity/v3/role_assignments?user.id=264c9bf15b9849a18b942d5d77559b9b&scope.domain.id=default&role.id=81457691f3a2456fa33358cb6a922d05 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:21.403699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923392c5-ba4b-44b5-b96c-4285531ad744 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:21.432401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:21.444912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998f1a3-0f3e-46f2-a2f5-56caeffae5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:21.445594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998f1a3-0f3e-46f2-a2f5-56caeffae5c3 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:21.445882 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998f1a3-0f3e-46f2-a2f5-56caeffae5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:21.446172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998f1a3-0f3e-46f2-a2f5-56caeffae5c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:21.447119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998f1a3-0f3e-46f2-a2f5-56caeffae5c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:21.448427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998f1a3-0f3e-46f2-a2f5-56caeffae5c3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:21.536157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 28/57] 10.222.0.88 () {60 vars in 1140 bytes} [Mon Jul 18 16:29:21 2022] POST /identity/v3/projects => generated 298 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:29:23.569368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c6ef-c8db-4d21-955e-84185a7ad558 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:23.569914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c6ef-c8db-4d21-955e-84185a7ad558 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:23.569914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14c6ef-c8db-4d21-955e-84185a7ad558 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:23.571313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 30/58] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:23.576533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236bd5b-5717-40e3-8c5c-12df59803954 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:23.576928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236bd5b-5717-40e3-8c5c-12df59803954 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:23.577144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236bd5b-5717-40e3-8c5c-12df59803954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:23.594993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9236bd5b-5717-40e3-8c5c-12df59803954 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:23.596246 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9236bd5b-5717-40e3-8c5c-12df59803954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:23.664569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 29/59] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:23.689276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14c6ef-c8db-4d21-955e-84185a7ad558 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:23.793257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14c6ef-c8db-4d21-955e-84185a7ad558 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:23.828673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7701c2-497c-4555-90bd-41f16fcad0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:23.829418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7701c2-497c-4555-90bd-41f16fcad0bd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:23.829706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7701c2-497c-4555-90bd-41f16fcad0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:23.829983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7701c2-497c-4555-90bd-41f16fcad0bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:23.831279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7701c2-497c-4555-90bd-41f16fcad0bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:23.832762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7701c2-497c-4555-90bd-41f16fcad0bd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:23.834462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 31/60] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:23.840043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236bd5b-5717-40e3-8c5c-12df59803954 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:23.940185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236bd5b-5717-40e3-8c5c-12df59803954 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:23.947677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:23.973997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145dff1d-debc-4b52-b635-c11a38ddb627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:23.974672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dff1d-debc-4b52-b635-c11a38ddb627 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:23.974960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dff1d-debc-4b52-b635-c11a38ddb627 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:23.975236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145dff1d-debc-4b52-b635-c11a38ddb627 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:23.975981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dff1d-debc-4b52-b635-c11a38ddb627 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:23.977490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145dff1d-debc-4b52-b635-c11a38ddb627 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:24.124380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 30/61] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:29:23 2022] POST /identity/v3/users => generated 275 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:29:27.271461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cab8b9-ed77-4559-b695-30b02f394293 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.272064 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cab8b9-ed77-4559-b695-30b02f394293 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.272064 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cab8b9-ed77-4559-b695-30b02f394293 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.273643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 32/62] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:27.279113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95343a-9e46-4d2b-9ec3-c9c8ef96a723 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.279584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95343a-9e46-4d2b-9ec3-c9c8ef96a723 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.279663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95343a-9e46-4d2b-9ec3-c9c8ef96a723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.315731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ac95343a-9e46-4d2b-9ec3-c9c8ef96a723 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:27.316971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ac95343a-9e46-4d2b-9ec3-c9c8ef96a723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:27.400356 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed415f1f-002f-441f-8024-ee8dbd349af6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.401053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed415f1f-002f-441f-8024-ee8dbd349af6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.401555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed415f1f-002f-441f-8024-ee8dbd349af6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.403251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 33/63] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:27.409464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c9700-7f66-4d9c-8782-03b322bb189a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.410053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c9700-7f66-4d9c-8782-03b322bb189a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.410561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c9700-7f66-4d9c-8782-03b322bb189a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.410662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 31/64] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:27.445958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9a6c9700-7f66-4d9c-8782-03b322bb189a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:27.447545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9a6c9700-7f66-4d9c-8782-03b322bb189a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:27.459847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95343a-9e46-4d2b-9ec3-c9c8ef96a723 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:27.462282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:27.572492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 34/65] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:27.608002 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c9700-7f66-4d9c-8782-03b322bb189a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:27.637261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:27.647326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95343a-9e46-4d2b-9ec3-c9c8ef96a723 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:27.689029 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e4992-1ebf-47f0-acd0-0bf9c083df2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:27.690453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e4992-1ebf-47f0-acd0-0bf9c083df2c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.690453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e4992-1ebf-47f0-acd0-0bf9c083df2c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.690453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e4992-1ebf-47f0-acd0-0bf9c083df2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.702998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e4992-1ebf-47f0-acd0-0bf9c083df2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c34e4992-1ebf-47f0-acd0-0bf9c083df2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:27.712329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:27.713276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:27.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 32/66] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:29:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:27.715243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1a8d-ef4e-4c32-80bc-a9d7d58b116d None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.715549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1a8d-ef4e-4c32-80bc-a9d7d58b116d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.716015 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1a8d-ef4e-4c32-80bc-a9d7d58b116d None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.717700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 33/67] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:27.719071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acb51c-abb4-4226-803f-ffcf9f4178cf None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.719364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acb51c-abb4-4226-803f-ffcf9f4178cf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.719644 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acb51c-abb4-4226-803f-ffcf9f4178cf None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.720903 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 34/68] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:27.721754 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63acb51c-abb4-4226-803f-ffcf9f4178cf None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:27.750269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c9700-7f66-4d9c-8782-03b322bb189a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:27.789077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cde5b-7457-4866-afbe-4b2c1f88ddc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:27.789892 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cde5b-7457-4866-afbe-4b2c1f88ddc6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.790182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cde5b-7457-4866-afbe-4b2c1f88ddc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.790465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cde5b-7457-4866-afbe-4b2c1f88ddc6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.801786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cde5b-7457-4866-afbe-4b2c1f88ddc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a22cde5b-7457-4866-afbe-4b2c1f88ddc6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:27.809630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:27.810476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:27.811233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 35/69] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:27.812921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83741bf-25a1-4b64-895c-1820583e1e3f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.812921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83741bf-25a1-4b64-895c-1820583e1e3f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.812921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83741bf-25a1-4b64-895c-1820583e1e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.825519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:27.831742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c83741bf-25a1-4b64-895c-1820583e1e3f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:27.832971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c83741bf-25a1-4b64-895c-1820583e1e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:27.845042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63acb51c-abb4-4226-803f-ffcf9f4178cf None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:27.883184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32738b0a-02c4-4b33-bd47-2815a704d2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:27.883983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32738b0a-02c4-4b33-bd47-2815a704d2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.884273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32738b0a-02c4-4b33-bd47-2815a704d2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.884671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32738b0a-02c4-4b33-bd47-2815a704d2d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.885481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32738b0a-02c4-4b33-bd47-2815a704d2d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:27.885794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32738b0a-02c4-4b33-bd47-2815a704d2d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:27.887329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32738b0a-02c4-4b33-bd47-2815a704d2d0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:27.897859 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 35/70] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:27.899254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4488-2fb2-42f2-a855-6630d420c3af None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.900697 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4488-2fb2-42f2-a855-6630d420c3af None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.900697 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd4488-2fb2-42f2-a855-6630d420c3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.908580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 36/71] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:27.908580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83741bf-25a1-4b64-895c-1820583e1e3f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:27.921315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-62bd4488-2fb2-42f2-a855-6630d420c3af None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:27.921315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-62bd4488-2fb2-42f2-a855-6630d420c3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:27.930256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:27.991232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 36/72] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:27.992600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8c1ac-c4ad-463d-a89b-30dff452153f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:27.993001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8c1ac-c4ad-463d-a89b-30dff452153f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:27.993305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8c1ac-c4ad-463d-a89b-30dff452153f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:27.994929 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 37/73] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:27.997651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8c1ac-c4ad-463d-a89b-30dff452153f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:28.023718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83741bf-25a1-4b64-895c-1820583e1e3f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:28.058855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9271f8-28c9-4c9c-a46b-ec106c7e4ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:28.059524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9271f8-28c9-4c9c-a46b-ec106c7e4ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:28.059811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9271f8-28c9-4c9c-a46b-ec106c7e4ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:28.060089 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9271f8-28c9-4c9c-a46b-ec106c7e4ad9 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:28.060846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9271f8-28c9-4c9c-a46b-ec106c7e4ad9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:28.061188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9271f8-28c9-4c9c-a46b-ec106c7e4ad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:28.062765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9271f8-28c9-4c9c-a46b-ec106c7e4ad9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:28.065102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:28.071606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 37/74] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:28.072414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9271f8-28c9-4c9c-a46b-ec106c7e4ad9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:28.105614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8c1ac-c4ad-463d-a89b-30dff452153f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:28.140726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a43e1-6004-494f-8907-87b80dccea39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:28.141633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a43e1-6004-494f-8907-87b80dccea39 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:28.141971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a43e1-6004-494f-8907-87b80dccea39 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:28.142312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a43e1-6004-494f-8907-87b80dccea39 None admin] PATH_INFO: `/v3/users/264c9bf15b9849a18b942d5d77559b9b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:28.143581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a43e1-6004-494f-8907-87b80dccea39 None admin] RBAC: Authorizing `identity:get_user(user_id=264c9bf15b9849a18b942d5d77559b9b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:28.145092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a43e1-6004-494f-8907-87b80dccea39 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:28.146827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 38/75] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:27 2022] GET /identity/v3/users/264c9bf15b9849a18b942d5d77559b9b => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:28.153224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e832d7-a066-4385-9e32-9062e65afb64 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:28.153224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e832d7-a066-4385-9e32-9062e65afb64 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:28.153224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e832d7-a066-4385-9e32-9062e65afb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:28.168294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-42e832d7-a066-4385-9e32-9062e65afb64 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:28.169722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-42e832d7-a066-4385-9e32-9062e65afb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:28.177670 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9271f8-28c9-4c9c-a46b-ec106c7e4ad9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:28.177831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:28.213899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2198ae-2bbf-4b3a-a288-63891ea12b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:28.214722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2198ae-2bbf-4b3a-a288-63891ea12b1d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:28.215012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2198ae-2bbf-4b3a-a288-63891ea12b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:28.215295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2198ae-2bbf-4b3a-a288-63891ea12b1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:28.225958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2198ae-2bbf-4b3a-a288-63891ea12b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-da2198ae-2bbf-4b3a-a288-63891ea12b1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:28.234008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:28.235151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:28.235881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 38/76] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:29:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:28.236684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2198ae-2bbf-4b3a-a288-63891ea12b1d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:28.242268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 39/77] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:28.243075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e832d7-a066-4385-9e32-9062e65afb64 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:28.302745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:28.346960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2198ae-2bbf-4b3a-a288-63891ea12b1d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:28.348253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e832d7-a066-4385-9e32-9062e65afb64 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:28.381518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80de383-a4e8-4732-a17f-4dbaecf164e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:28.382236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80de383-a4e8-4732-a17f-4dbaecf164e8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:28.382521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80de383-a4e8-4732-a17f-4dbaecf164e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:28.382801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80de383-a4e8-4732-a17f-4dbaecf164e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:28.384528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcdf36-01ad-4bcb-beef-e50551a48e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:28.385332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcdf36-01ad-4bcb-beef-e50551a48e00 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:28.385617 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcdf36-01ad-4bcb-beef-e50551a48e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:28.385894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcdf36-01ad-4bcb-beef-e50551a48e00 None admin] PATH_INFO: `/v3/users/72ac367241b34a4bafbea8748bd53095` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:28.393055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80de383-a4e8-4732-a17f-4dbaecf164e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:28.398843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f80de383-a4e8-4732-a17f-4dbaecf164e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:28.400752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:28.401617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:28.402572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:28.403358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dcdf36-01ad-4bcb-beef-e50551a48e00 None admin] RBAC: Authorizing `identity:get_user(user_id=72ac367241b34a4bafbea8748bd53095)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:28.403438 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 39/78] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:28.403519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dcdf36-01ad-4bcb-beef-e50551a48e00 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:28.404751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80de383-a4e8-4732-a17f-4dbaecf164e8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:28.409054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 40/79] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:28 2022] GET /identity/v3/users/72ac367241b34a4bafbea8748bd53095 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:28.415195 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcdf36-01ad-4bcb-beef-e50551a48e00 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:28.514283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80de383-a4e8-4732-a17f-4dbaecf164e8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:28.520414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcdf36-01ad-4bcb-beef-e50551a48e00 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:28.532720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:28.549399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230fb4f-e5d9-4600-8b92-86e485c78300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:28.550056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230fb4f-e5d9-4600-8b92-86e485c78300 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:28.550346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230fb4f-e5d9-4600-8b92-86e485c78300 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:28.550632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230fb4f-e5d9-4600-8b92-86e485c78300 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:28.554592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7c154-855b-4c9c-bdd0-60835cd8daa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:28.555264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7c154-855b-4c9c-bdd0-60835cd8daa5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:28.555554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7c154-855b-4c9c-bdd0-60835cd8daa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:28.555836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7c154-855b-4c9c-bdd0-60835cd8daa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:28.556597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7c154-855b-4c9c-bdd0-60835cd8daa5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:28.556930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7c154-855b-4c9c-bdd0-60835cd8daa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:28.558327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7c154-855b-4c9c-bdd0-60835cd8daa5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:28.566615 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 41/80] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:29:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:28.567428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7c154-855b-4c9c-bdd0-60835cd8daa5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:28.619605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230fb4f-e5d9-4600-8b92-86e485c78300 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:28.621118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230fb4f-e5d9-4600-8b92-86e485c78300 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:28.629361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:28.679134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7c154-855b-4c9c-bdd0-60835cd8daa5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:28.702061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ce837-d38a-4db4-8d5c-f88eafeadeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:28.703398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ce837-d38a-4db4-8d5c-f88eafeadeab None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:28.703679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ce837-d38a-4db4-8d5c-f88eafeadeab None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:28.703990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ce837-d38a-4db4-8d5c-f88eafeadeab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:28.705225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ce837-d38a-4db4-8d5c-f88eafeadeab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:28.706689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ce837-d38a-4db4-8d5c-f88eafeadeab None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:28.708341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 42/81] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:28.709825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ce837-d38a-4db4-8d5c-f88eafeadeab None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:28.722845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 40/82] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:28.723693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230fb4f-e5d9-4600-8b92-86e485c78300 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:28.742321 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:28.813182 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ce837-d38a-4db4-8d5c-f88eafeadeab None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:28.824429 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230fb4f-e5d9-4600-8b92-86e485c78300 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:28.843677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:28.851237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce252fd-95e0-41b2-ab86-1ee781d62597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:28.856453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce252fd-95e0-41b2-ab86-1ee781d62597 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:28.856453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce252fd-95e0-41b2-ab86-1ee781d62597 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:28.856453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce252fd-95e0-41b2-ab86-1ee781d62597 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:28.856453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce252fd-95e0-41b2-ab86-1ee781d62597 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:28.856453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce252fd-95e0-41b2-ab86-1ee781d62597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:28.856453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce252fd-95e0-41b2-ab86-1ee781d62597 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:28.860641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d9607-e624-4492-8c58-907264eca54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:28.861332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d9607-e624-4492-8c58-907264eca54e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:28.861619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d9607-e624-4492-8c58-907264eca54e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:28.861897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d9607-e624-4492-8c58-907264eca54e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:28.863964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 43/83] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:28.864787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce252fd-95e0-41b2-ab86-1ee781d62597 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:28.938796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d9607-e624-4492-8c58-907264eca54e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:28.940314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d9607-e624-4492-8c58-907264eca54e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:28.958639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:28.968559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce252fd-95e0-41b2-ab86-1ee781d62597 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:29.003214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86b86d-cd48-4089-bbd0-8507c19f71f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:29.004089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86b86d-cd48-4089-bbd0-8507c19f71f5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:29.004375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86b86d-cd48-4089-bbd0-8507c19f71f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:29.004652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86b86d-cd48-4089-bbd0-8507c19f71f5 None admin] PATH_INFO: `/v3/users/72ac367241b34a4bafbea8748bd53095` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:29.005917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86b86d-cd48-4089-bbd0-8507c19f71f5 None admin] RBAC: Authorizing `identity:get_user(user_id=72ac367241b34a4bafbea8748bd53095)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:29.007394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86b86d-cd48-4089-bbd0-8507c19f71f5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:29.009132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 44/84] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:28 2022] GET /identity/v3/users/72ac367241b34a4bafbea8748bd53095 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:29.010267 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86b86d-cd48-4089-bbd0-8507c19f71f5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:29.047879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 41/85] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:29.049945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d9607-e624-4492-8c58-907264eca54e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:29.087571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:29.130177 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86b86d-cd48-4089-bbd0-8507c19f71f5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:29.150704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d9607-e624-4492-8c58-907264eca54e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:29.165035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ffa88-a75d-48bd-9a1f-1218d60792ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:29.165603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ffa88-a75d-48bd-9a1f-1218d60792ab None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:29.165891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ffa88-a75d-48bd-9a1f-1218d60792ab None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:29.166170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ffa88-a75d-48bd-9a1f-1218d60792ab None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:29.166884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ffa88-a75d-48bd-9a1f-1218d60792ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:29.168192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:29.168399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ffa88-a75d-48bd-9a1f-1218d60792ab None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:29.185030 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62140a-8f96-4073-b2ea-64c35952c991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:29.185839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62140a-8f96-4073-b2ea-64c35952c991 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:29.186117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62140a-8f96-4073-b2ea-64c35952c991 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:29.186392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62140a-8f96-4073-b2ea-64c35952c991 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:29.187653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62140a-8f96-4073-b2ea-64c35952c991 None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:29.189122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62140a-8f96-4073-b2ea-64c35952c991 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:29.190815 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 42/86] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:29 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:29.191596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62140a-8f96-4073-b2ea-64c35952c991 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:29.290324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62140a-8f96-4073-b2ea-64c35952c991 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:29.295215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 45/87] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:29:29 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:29:29.296059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ffa88-a75d-48bd-9a1f-1218d60792ab None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:29.331823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34915dba-410a-4c9e-ad68-e95fe75e1d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:29.333719 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915dba-410a-4c9e-ad68-e95fe75e1d45 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:29.334013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915dba-410a-4c9e-ad68-e95fe75e1d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:29.334294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915dba-410a-4c9e-ad68-e95fe75e1d45 None admin] PATH_INFO: `/v3/users/72ac367241b34a4bafbea8748bd53095` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:29.335644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34915dba-410a-4c9e-ad68-e95fe75e1d45 None admin] RBAC: Authorizing `identity:get_user(user_id=72ac367241b34a4bafbea8748bd53095)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:29.337282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34915dba-410a-4c9e-ad68-e95fe75e1d45 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:29.344416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 43/88] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:29 2022] GET /identity/v3/users/72ac367241b34a4bafbea8748bd53095 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:29.344416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34915dba-410a-4c9e-ad68-e95fe75e1d45 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:29.410664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:29.425476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ffa88-a75d-48bd-9a1f-1218d60792ab None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:29.445015 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:29.467346 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7033-5b3b-4b08-9156-dee31a4be9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:29.468557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7033-5b3b-4b08-9156-dee31a4be9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:29.468845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7033-5b3b-4b08-9156-dee31a4be9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:29.469157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7033-5b3b-4b08-9156-dee31a4be9a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:29.470225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34915dba-410a-4c9e-ad68-e95fe75e1d45 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:29.509245 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5f9f7-ac2f-4a85-9ef6-15908f48fb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:29.509893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5f9f7-ac2f-4a85-9ef6-15908f48fb02 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:29.510183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5f9f7-ac2f-4a85-9ef6-15908f48fb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:29.511129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5f9f7-ac2f-4a85-9ef6-15908f48fb02 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:29.511690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5f9f7-ac2f-4a85-9ef6-15908f48fb02 None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:29.513185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5f9f7-ac2f-4a85-9ef6-15908f48fb02 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:29.516044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 44/89] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:29 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:29.521644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5f9f7-ac2f-4a85-9ef6-15908f48fb02 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:29.562723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee7033-5b3b-4b08-9156-dee31a4be9a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:29.564336 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee7033-5b3b-4b08-9156-dee31a4be9a0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:29.650793 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5f9f7-ac2f-4a85-9ef6-15908f48fb02 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:29.676167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:29.690838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03898dc8-0ab1-4d04-97fa-4d3bc75f214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:29.693238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03898dc8-0ab1-4d04-97fa-4d3bc75f214d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:29.693238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03898dc8-0ab1-4d04-97fa-4d3bc75f214d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:29.693500 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03898dc8-0ab1-4d04-97fa-4d3bc75f214d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:29.694885 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03898dc8-0ab1-4d04-97fa-4d3bc75f214d None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf, user.id=264c9bf15b9849a18b942d5d77559b9b) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:29.695260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03898dc8-0ab1-4d04-97fa-4d3bc75f214d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:29.696695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03898dc8-0ab1-4d04-97fa-4d3bc75f214d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:29.704746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 45/90] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:29 2022] GET /identity/v3/role_assignments?user.id=264c9bf15b9849a18b942d5d77559b9b&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=81457691f3a2456fa33358cb6a922d05 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:29.705763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03898dc8-0ab1-4d04-97fa-4d3bc75f214d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:29.713396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 46/91] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:29.714062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7033-5b3b-4b08-9156-dee31a4be9a0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:29.739088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:29.832613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7033-5b3b-4b08-9156-dee31a4be9a0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:29.868555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03898dc8-0ab1-4d04-97fa-4d3bc75f214d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:29.887581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e257c3-92aa-4782-8da1-3115e165a4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:29.888272 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e257c3-92aa-4782-8da1-3115e165a4df None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:29.888565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e257c3-92aa-4782-8da1-3115e165a4df None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:29.888847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e257c3-92aa-4782-8da1-3115e165a4df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:29.890412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e257c3-92aa-4782-8da1-3115e165a4df None admin] RBAC: Adding query filter params (role.id=3b2e1fad42f441e9a61c142477a50b11, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf, user.id=72ac367241b34a4bafbea8748bd53095) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:29.890683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e257c3-92aa-4782-8da1-3115e165a4df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:29.892192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e257c3-92aa-4782-8da1-3115e165a4df None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:29.901487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 47/92] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:29 2022] GET /identity/v3/role_assignments?user.id=72ac367241b34a4bafbea8748bd53095&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:29.902363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e257c3-92aa-4782-8da1-3115e165a4df None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:29.974553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc030318-6d7a-4e8b-be68-f9a2dc3c84a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:29.975159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc030318-6d7a-4e8b-be68-f9a2dc3c84a4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:29.975450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc030318-6d7a-4e8b-be68-f9a2dc3c84a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:29.975736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc030318-6d7a-4e8b-be68-f9a2dc3c84a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:30.018411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:30.063603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e257c3-92aa-4782-8da1-3115e165a4df None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:30.120384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:30.153706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca3bb1-ebe1-4818-a8cf-8caa645d1b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:30.154610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca3bb1-ebe1-4818-a8cf-8caa645d1b0b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:30.154902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca3bb1-ebe1-4818-a8cf-8caa645d1b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:30.155185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca3bb1-ebe1-4818-a8cf-8caa645d1b0b None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:30.161797 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca3bb1-ebe1-4818-a8cf-8caa645d1b0b None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:30.163316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca3bb1-ebe1-4818-a8cf-8caa645d1b0b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:30.165348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 48/93] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:29 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:30.168994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc030318-6d7a-4e8b-be68-f9a2dc3c84a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:30.168994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc030318-6d7a-4e8b-be68-f9a2dc3c84a4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:30.174068 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca3bb1-ebe1-4818-a8cf-8caa645d1b0b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:30.283622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 46/94] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:30.289585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc030318-6d7a-4e8b-be68-f9a2dc3c84a4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:30.389302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:30.401489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc030318-6d7a-4e8b-be68-f9a2dc3c84a4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:30.447963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665a0ca-afd4-4e1b-94ea-38a513c4879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:30.448843 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665a0ca-afd4-4e1b-94ea-38a513c4879c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:30.449241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665a0ca-afd4-4e1b-94ea-38a513c4879c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:30.449523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665a0ca-afd4-4e1b-94ea-38a513c4879c None admin] PATH_INFO: `/v3/projects/77434394fa334e2cb6914227f2c2d09d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:30.453043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665a0ca-afd4-4e1b-94ea-38a513c4879c None admin] RBAC: Authorizing `identity:get_project(project_id=77434394fa334e2cb6914227f2c2d09d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:30.454577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665a0ca-afd4-4e1b-94ea-38a513c4879c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:30.456324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 47/95] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:30 2022] GET /identity/v3/projects/77434394fa334e2cb6914227f2c2d09d => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:30.463251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665a0ca-afd4-4e1b-94ea-38a513c4879c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:30.470843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca3bb1-ebe1-4818-a8cf-8caa645d1b0b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:30.543779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e7ddb-c0e8-43f2-84d5-9c4fb63bdebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:30.544458 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e7ddb-c0e8-43f2-84d5-9c4fb63bdebc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:30.544750 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e7ddb-c0e8-43f2-84d5-9c4fb63bdebc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:30.545087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e7ddb-c0e8-43f2-84d5-9c4fb63bdebc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:30.546589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e7ddb-c0e8-43f2-84d5-9c4fb63bdebc None admin] RBAC: Adding query filter params (role.id=c465d586f0ec4b59815b23087d757fd2, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf, user.id=72ac367241b34a4bafbea8748bd53095) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:30.546834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e7ddb-c0e8-43f2-84d5-9c4fb63bdebc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:30.548417 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e7ddb-c0e8-43f2-84d5-9c4fb63bdebc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:30.561513 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 49/96] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:30 2022] GET /identity/v3/role_assignments?user.id=72ac367241b34a4bafbea8748bd53095&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=c465d586f0ec4b59815b23087d757fd2 => generated 259 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:30.672596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665a0ca-afd4-4e1b-94ea-38a513c4879c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:30.732348 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d4a3f-93f8-4c53-9afe-e43c61c76c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:30.733067 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d4a3f-93f8-4c53-9afe-e43c61c76c7e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:30.733380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d4a3f-93f8-4c53-9afe-e43c61c76c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:30.734200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d4a3f-93f8-4c53-9afe-e43c61c76c7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:30.735065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d4a3f-93f8-4c53-9afe-e43c61c76c7e None admin] RBAC: Adding query filter params (role.id=3b2e1fad42f441e9a61c142477a50b11, scope.project.id=77434394fa334e2cb6914227f2c2d09d, user.id=72ac367241b34a4bafbea8748bd53095) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:30.735376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d4a3f-93f8-4c53-9afe-e43c61c76c7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:30.736834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d4a3f-93f8-4c53-9afe-e43c61c76c7e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:30.746863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 48/97] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:30 2022] GET /identity/v3/role_assignments?user.id=72ac367241b34a4bafbea8748bd53095&scope.project.id=77434394fa334e2cb6914227f2c2d09d&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:32.493396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afed9a-2cf0-45de-b082-4a5d73e90d02 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:32.494159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afed9a-2cf0-45de-b082-4a5d73e90d02 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:32.494232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afed9a-2cf0-45de-b082-4a5d73e90d02 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:32.495791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 50/98] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:32.500979 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da060a2e-5487-4dfa-933e-2e1bd377e923 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:32.501421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da060a2e-5487-4dfa-933e-2e1bd377e923 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:32.501614 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da060a2e-5487-4dfa-933e-2e1bd377e923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:32.519225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-da060a2e-5487-4dfa-933e-2e1bd377e923 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:32.520445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-da060a2e-5487-4dfa-933e-2e1bd377e923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:32.535399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:32.603270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 49/99] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:32.646735 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afed9a-2cf0-45de-b082-4a5d73e90d02 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:32.668471 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:32.776216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afed9a-2cf0-45de-b082-4a5d73e90d02 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:32.814366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf50ed-8e25-4c28-90b9-4aa8521c03d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:32.815203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf50ed-8e25-4c28-90b9-4aa8521c03d9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:32.815493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf50ed-8e25-4c28-90b9-4aa8521c03d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:32.815913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf50ed-8e25-4c28-90b9-4aa8521c03d9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:32.827316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf50ed-8e25-4c28-90b9-4aa8521c03d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6fdf50ed-8e25-4c28-90b9-4aa8521c03d9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:32.835481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:32.836323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:32.837135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 51/100] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:32.843458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da060a2e-5487-4dfa-933e-2e1bd377e923 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:32.971595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a7575-4658-4315-b2ba-b29c64dc489f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:32.971898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a7575-4658-4315-b2ba-b29c64dc489f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:32.972191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a7575-4658-4315-b2ba-b29c64dc489f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:32.973629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 52/101] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:32.974756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbba5c-14fd-40ac-ae92-a9650f827f64 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:32.975039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbba5c-14fd-40ac-ae92-a9650f827f64 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:32.975532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbba5c-14fd-40ac-ae92-a9650f827f64 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:32.981034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 53/102] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:32.982614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18840e-0735-429f-841f-bd65fa398eda None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:32.983112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18840e-0735-429f-841f-bd65fa398eda None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:32.983694 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18840e-0735-429f-841f-bd65fa398eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:32.998085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da060a2e-5487-4dfa-933e-2e1bd377e923 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:33.013023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fa18840e-0735-429f-841f-bd65fa398eda None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:33.015220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fa18840e-0735-429f-841f-bd65fa398eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:33.042102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8f484-29f0-4dac-b069-2562ecf27846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:33.042704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8f484-29f0-4dac-b069-2562ecf27846 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:33.043149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8f484-29f0-4dac-b069-2562ecf27846 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:33.043574 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8f484-29f0-4dac-b069-2562ecf27846 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:33.044475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8f484-29f0-4dac-b069-2562ecf27846 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:33.044952 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8f484-29f0-4dac-b069-2562ecf27846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:33.047044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8f484-29f0-4dac-b069-2562ecf27846 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:33.056248 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 50/103] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:33.063126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e0be9-3fe9-46dc-a03d-085ba81a895d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:33.063864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e0be9-3fe9-46dc-a03d-085ba81a895d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:33.065210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e0be9-3fe9-46dc-a03d-085ba81a895d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:33.072908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 51/104] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:33.075378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cf3ee-7360-4e3d-b947-2b0543a9350f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:33.076026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cf3ee-7360-4e3d-b947-2b0543a9350f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:33.076660 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cf3ee-7360-4e3d-b947-2b0543a9350f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:33.097772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 54/105] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:33.098900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18840e-0735-429f-841f-bd65fa398eda None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:33.101291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:33.102007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-cc1cf3ee-7360-4e3d-b947-2b0543a9350f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:33.103501 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cc1cf3ee-7360-4e3d-b947-2b0543a9350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:33.107241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:33.175522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 52/106] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:33.177230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac3ea7-086b-46f9-af9c-0f19dbc80060 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:33.177756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac3ea7-086b-46f9-af9c-0f19dbc80060 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:33.178238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac3ea7-086b-46f9-af9c-0f19dbc80060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:33.197317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a5ac3ea7-086b-46f9-af9c-0f19dbc80060 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:33.198810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a5ac3ea7-086b-46f9-af9c-0f19dbc80060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:33.212186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18840e-0735-429f-841f-bd65fa398eda None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:33.251016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6afa1-ac7f-4376-b105-a3a80ac23018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:33.251902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6afa1-ac7f-4376-b105-a3a80ac23018 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:33.252193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6afa1-ac7f-4376-b105-a3a80ac23018 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:33.252528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6afa1-ac7f-4376-b105-a3a80ac23018 None admin] PATH_INFO: `/v3/users/9dc916d276d640d7bbc8243bd1539ee1` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:33.268308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6afa1-ac7f-4376-b105-a3a80ac23018 None admin] RBAC: Authorizing `identity:get_user(user_id=9dc916d276d640d7bbc8243bd1539ee1)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:33.269838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6afa1-ac7f-4376-b105-a3a80ac23018 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:33.271578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 55/107] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:33 2022] GET /identity/v3/users/9dc916d276d640d7bbc8243bd1539ee1 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:33.276467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6afa1-ac7f-4376-b105-a3a80ac23018 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:33.277916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 53/108] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:33.282724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac3ea7-086b-46f9-af9c-0f19dbc80060 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:33.370203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:33.383378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:33.422158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac3ea7-086b-46f9-af9c-0f19dbc80060 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:33.429218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6afa1-ac7f-4376-b105-a3a80ac23018 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:33.472243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb631216-f631-4a92-ab4c-f7b448063bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:33.472855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88cc27-8873-41cf-8648-bdbbe253a99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:33.473253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb631216-f631-4a92-ab4c-f7b448063bcc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:33.473647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb631216-f631-4a92-ab4c-f7b448063bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:33.474005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb631216-f631-4a92-ab4c-f7b448063bcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:33.474425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88cc27-8873-41cf-8648-bdbbe253a99c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:33.474889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88cc27-8873-41cf-8648-bdbbe253a99c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:33.475331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88cc27-8873-41cf-8648-bdbbe253a99c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:33.475426 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb631216-f631-4a92-ab4c-f7b448063bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:33.476980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb631216-f631-4a92-ab4c-f7b448063bcc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:33.478815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 56/109] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:33.479707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb631216-f631-4a92-ab4c-f7b448063bcc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:33.494863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88cc27-8873-41cf-8648-bdbbe253a99c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1c88cc27-8873-41cf-8648-bdbbe253a99c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:33.506255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:33.507701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:33.508618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:33.511939 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 54/110] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:33.513171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88cc27-8873-41cf-8648-bdbbe253a99c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:33.612223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb631216-f631-4a92-ab4c-f7b448063bcc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:33.642819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:33.648970 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88cc27-8873-41cf-8648-bdbbe253a99c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:33.650990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555535a6-ac49-4506-8769-42687fb08ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:33.651662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555535a6-ac49-4506-8769-42687fb08ffb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:33.657384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555535a6-ac49-4506-8769-42687fb08ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:33.657384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555535a6-ac49-4506-8769-42687fb08ffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:33.689446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:33.698045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8293e0c-6ba9-44d8-bec9-98ae7a73062c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:33.699665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8293e0c-6ba9-44d8-bec9-98ae7a73062c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:33.703173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8293e0c-6ba9-44d8-bec9-98ae7a73062c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:33.703173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8293e0c-6ba9-44d8-bec9-98ae7a73062c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:33.719124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8293e0c-6ba9-44d8-bec9-98ae7a73062c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b8293e0c-6ba9-44d8-bec9-98ae7a73062c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:33.731900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:33.732966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:33.733860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:33.735362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555535a6-ac49-4506-8769-42687fb08ffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:33.736123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 55/111] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:29:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:33.737132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555535a6-ac49-4506-8769-42687fb08ffb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:33.737403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8293e0c-6ba9-44d8-bec9-98ae7a73062c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:33.856594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 57/112] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:33.857648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555535a6-ac49-4506-8769-42687fb08ffb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:33.878420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8293e0c-6ba9-44d8-bec9-98ae7a73062c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:33.893047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:33.915312 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f46dcc-04b0-4de6-aa59-84f366183f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:33.916022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f46dcc-04b0-4de6-aa59-84f366183f32 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:33.916350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f46dcc-04b0-4de6-aa59-84f366183f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:33.916905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f46dcc-04b0-4de6-aa59-84f366183f32 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:33.917404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f46dcc-04b0-4de6-aa59-84f366183f32 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:33.919021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f46dcc-04b0-4de6-aa59-84f366183f32 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:33.973822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555535a6-ac49-4506-8769-42687fb08ffb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:34.008208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868bbe7-6326-44b9-9ebb-8d4f3389fef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:34.008869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868bbe7-6326-44b9-9ebb-8d4f3389fef4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.009186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868bbe7-6326-44b9-9ebb-8d4f3389fef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.009455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868bbe7-6326-44b9-9ebb-8d4f3389fef4 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.010200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868bbe7-6326-44b9-9ebb-8d4f3389fef4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:34.010503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868bbe7-6326-44b9-9ebb-8d4f3389fef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:34.021009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 56/113] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:29:33 2022] POST /identity/v3/projects => generated 302 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:29:34.021009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f46dcc-04b0-4de6-aa59-84f366183f32 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:34.021288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868bbe7-6326-44b9-9ebb-8d4f3389fef4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:34.021288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 58/114] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:34.022863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868bbe7-6326-44b9-9ebb-8d4f3389fef4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:34.076698 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:34.136219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:34.147321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868bbe7-6326-44b9-9ebb-8d4f3389fef4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:34.148278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f46dcc-04b0-4de6-aa59-84f366183f32 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:34.188640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c542b-cf2f-45d4-8209-defc57c53112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:34.191780 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c542b-cf2f-45d4-8209-defc57c53112 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.191780 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c542b-cf2f-45d4-8209-defc57c53112 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.191780 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c542b-cf2f-45d4-8209-defc57c53112 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.192088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c542b-cf2f-45d4-8209-defc57c53112 None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:34.192831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad124c-c829-468c-91f2-1499b71f4ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:34.193734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c542b-cf2f-45d4-8209-defc57c53112 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:34.193968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad124c-c829-468c-91f2-1499b71f4ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.194395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad124c-c829-468c-91f2-1499b71f4ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.194678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad124c-c829-468c-91f2-1499b71f4ef2 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.195461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad124c-c829-468c-91f2-1499b71f4ef2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:34.195584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 59/115] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:34 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:34.196535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad124c-c829-468c-91f2-1499b71f4ef2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:34.196857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e03e5-b54b-460e-af85-ea09c45e2e2e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.197273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad124c-c829-468c-91f2-1499b71f4ef2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:34.197433 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e03e5-b54b-460e-af85-ea09c45e2e2e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.197861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e03e5-b54b-460e-af85-ea09c45e2e2e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.201420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 60/116] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:34.203468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a97a7-11f0-4860-ab0a-e0875878032d None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.204356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a97a7-11f0-4860-ab0a-e0875878032d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.204646 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a97a7-11f0-4860-ab0a-e0875878032d None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.206805 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 61/117] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:34.207339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 57/118] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:34.207593 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a97a7-11f0-4860-ab0a-e0875878032d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:34.208176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad124c-c829-468c-91f2-1499b71f4ef2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:34.322579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad124c-c829-468c-91f2-1499b71f4ef2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:34.325621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:34.326255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a97a7-11f0-4860-ab0a-e0875878032d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:34.359707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e16acf-e5b2-46f1-ad2c-e63af465d174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:34.361030 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e16acf-e5b2-46f1-ad2c-e63af465d174 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.361322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e16acf-e5b2-46f1-ad2c-e63af465d174 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.361602 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e16acf-e5b2-46f1-ad2c-e63af465d174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.362561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e16acf-e5b2-46f1-ad2c-e63af465d174 None admin] RBAC: Adding query filter params (role.id=4f99bdbe513b4500b96f9c530dac52ed, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf, user.id=9dc916d276d640d7bbc8243bd1539ee1) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:34.362742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3e63e-545b-4ebe-9c1c-f21ae78c3ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:34.362931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e16acf-e5b2-46f1-ad2c-e63af465d174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:34.363637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3e63e-545b-4ebe-9c1c-f21ae78c3ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.363928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3e63e-545b-4ebe-9c1c-f21ae78c3ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.364211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3e63e-545b-4ebe-9c1c-f21ae78c3ff8 None admin] PATH_INFO: `/v3/users/72ac367241b34a4bafbea8748bd53095` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.364363 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e16acf-e5b2-46f1-ad2c-e63af465d174 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:34.365516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3e63e-545b-4ebe-9c1c-f21ae78c3ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=72ac367241b34a4bafbea8748bd53095)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:34.367178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3e63e-545b-4ebe-9c1c-f21ae78c3ff8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:34.377487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 62/119] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:34 2022] GET /identity/v3/users/72ac367241b34a4bafbea8748bd53095 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:34.377678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 58/120] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:34 2022] GET /identity/v3/role_assignments?user.id=9dc916d276d640d7bbc8243bd1539ee1&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=4f99bdbe513b4500b96f9c530dac52ed => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:34.377756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664aa01-144a-4206-b8cc-656a4c5ca0fa None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.377756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664aa01-144a-4206-b8cc-656a4c5ca0fa None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.377756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664aa01-144a-4206-b8cc-656a4c5ca0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.377917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d5e12-6d76-42c4-afde-ccafba189800 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.377917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d5e12-6d76-42c4-afde-ccafba189800 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.377917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d5e12-6d76-42c4-afde-ccafba189800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.394234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2664aa01-144a-4206-b8cc-656a4c5ca0fa None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:34.395699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2664aa01-144a-4206-b8cc-656a4c5ca0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:34.395954 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b70d5e12-6d76-42c4-afde-ccafba189800 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:34.397291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b70d5e12-6d76-42c4-afde-ccafba189800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:34.422672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:34.486849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 59/121] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:34.487592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 63/122] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:34.488603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664aa01-144a-4206-b8cc-656a4c5ca0fa None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:34.493648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d5e12-6d76-42c4-afde-ccafba189800 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:34.628428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:34.632157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664aa01-144a-4206-b8cc-656a4c5ca0fa None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:34.632557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d5e12-6d76-42c4-afde-ccafba189800 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:34.674120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8ba99-6304-478a-b08a-73062a1e04f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:34.675751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8ba99-6304-478a-b08a-73062a1e04f6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.676290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8ba99-6304-478a-b08a-73062a1e04f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.676723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8ba99-6304-478a-b08a-73062a1e04f6 None admin] PATH_INFO: `/v3/users/264c9bf15b9849a18b942d5d77559b9b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.678451 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8ba99-6304-478a-b08a-73062a1e04f6 None admin] RBAC: Authorizing `identity:get_user(user_id=264c9bf15b9849a18b942d5d77559b9b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:34.678988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56226d5-35d4-4ec1-9eff-b29004436fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:34.679978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56226d5-35d4-4ec1-9eff-b29004436fcc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.680406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56226d5-35d4-4ec1-9eff-b29004436fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.680546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8ba99-6304-478a-b08a-73062a1e04f6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:34.680869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56226d5-35d4-4ec1-9eff-b29004436fcc None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.682229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56226d5-35d4-4ec1-9eff-b29004436fcc None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:34.682801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 60/123] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:34 2022] GET /identity/v3/users/264c9bf15b9849a18b942d5d77559b9b => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:34.683761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8ba99-6304-478a-b08a-73062a1e04f6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:34.689938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56226d5-35d4-4ec1-9eff-b29004436fcc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:34.691894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 64/124] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:34 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:34.692771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56226d5-35d4-4ec1-9eff-b29004436fcc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:34.721875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:34.814832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56226d5-35d4-4ec1-9eff-b29004436fcc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:34.816473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8ba99-6304-478a-b08a-73062a1e04f6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:34.852988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d7d24-976a-4942-ac46-56cf7fd9b0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:34.853809 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d7d24-976a-4942-ac46-56cf7fd9b0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.854047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f63b3-4725-4ea7-a245-8446a8fb7772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:34.854172 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d7d24-976a-4942-ac46-56cf7fd9b0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.854354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d7d24-976a-4942-ac46-56cf7fd9b0e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.854998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f63b3-4725-4ea7-a245-8446a8fb7772 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:34.855292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f63b3-4725-4ea7-a245-8446a8fb7772 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:34.855574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f63b3-4725-4ea7-a245-8446a8fb7772 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:34.866265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d7d24-976a-4942-ac46-56cf7fd9b0e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:34.868052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f63b3-4725-4ea7-a245-8446a8fb7772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4c8d7d24-976a-4942-ac46-56cf7fd9b0e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:34.875676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:34.876545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:34.877713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9f5f63b3-4725-4ea7-a245-8446a8fb7772 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:34.878504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:34.879970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:34.881367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:34.884474 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 61/125] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:29:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:34.884609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 65/126] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:34.884609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f63b3-4725-4ea7-a245-8446a8fb7772 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:34.884845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d7d24-976a-4942-ac46-56cf7fd9b0e1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:34.916180 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:34.986785 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:35.004400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f63b3-4725-4ea7-a245-8446a8fb7772 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:35.026985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d7d24-976a-4942-ac46-56cf7fd9b0e1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:35.041308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f53fba-bebf-425f-8bdb-f0eb31fbae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:35.042316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f53fba-bebf-425f-8bdb-f0eb31fbae27 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:35.042954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f53fba-bebf-425f-8bdb-f0eb31fbae27 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:35.042954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f53fba-bebf-425f-8bdb-f0eb31fbae27 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/users/72ac367241b34a4bafbea8748bd53095/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:35.045243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f53fba-bebf-425f-8bdb-f0eb31fbae27 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c6a41d535fe491c81d7c8ea25c933bf, user_id=72ac367241b34a4bafbea8748bd53095, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:35.047440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f53fba-bebf-425f-8bdb-f0eb31fbae27 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:35.070146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d4306-3d7c-4734-be2e-c820e7a98e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:35.071036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d4306-3d7c-4734-be2e-c820e7a98e9f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:35.071483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d4306-3d7c-4734-be2e-c820e7a98e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:35.071771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d4306-3d7c-4734-be2e-c820e7a98e9f None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:35.073025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d4306-3d7c-4734-be2e-c820e7a98e9f None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:35.074520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d4306-3d7c-4734-be2e-c820e7a98e9f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:35.076523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 62/127] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:34 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:35.077816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d4306-3d7c-4734-be2e-c820e7a98e9f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:35.156920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 66/128] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:29:34 2022] PUT /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/users/72ac367241b34a4bafbea8748bd53095/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:29:35.156920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f53fba-bebf-425f-8bdb-f0eb31fbae27 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:35.248526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:35.264220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d4306-3d7c-4734-be2e-c820e7a98e9f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:35.331365 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f53fba-bebf-425f-8bdb-f0eb31fbae27 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:35.355608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f2e96-e019-400d-ab61-556c815f20a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:35.356915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f2e96-e019-400d-ab61-556c815f20a7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:35.357263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f2e96-e019-400d-ab61-556c815f20a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:35.357589 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f2e96-e019-400d-ab61-556c815f20a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:35.359037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f2e96-e019-400d-ab61-556c815f20a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:35.359037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f2e96-e019-400d-ab61-556c815f20a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:35.360803 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f2e96-e019-400d-ab61-556c815f20a7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:35.372842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 63/129] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:29:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:35.373694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f2e96-e019-400d-ab61-556c815f20a7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:35.392179 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db01754-307e-4f3b-95b5-c38cb110eaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:35.394032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db01754-307e-4f3b-95b5-c38cb110eaaa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:35.394531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db01754-307e-4f3b-95b5-c38cb110eaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:35.395012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db01754-307e-4f3b-95b5-c38cb110eaaa None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:35.395992 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db01754-307e-4f3b-95b5-c38cb110eaaa None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:35.396609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db01754-307e-4f3b-95b5-c38cb110eaaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:35.398289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db01754-307e-4f3b-95b5-c38cb110eaaa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:35.410055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 67/130] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:35.411185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db01754-307e-4f3b-95b5-c38cb110eaaa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:35.426829 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:35.502194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f2e96-e019-400d-ab61-556c815f20a7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:35.542553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316c00d-d2ba-48ff-b509-c69ed4002076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:35.543641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316c00d-d2ba-48ff-b509-c69ed4002076 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:35.543932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316c00d-d2ba-48ff-b509-c69ed4002076 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:35.544210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316c00d-d2ba-48ff-b509-c69ed4002076 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/users/264c9bf15b9849a18b942d5d77559b9b/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:35.546460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316c00d-d2ba-48ff-b509-c69ed4002076 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c6a41d535fe491c81d7c8ea25c933bf, user_id=264c9bf15b9849a18b942d5d77559b9b, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:35.547900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316c00d-d2ba-48ff-b509-c69ed4002076 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:35.589305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db01754-307e-4f3b-95b5-c38cb110eaaa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:35.634946 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dee3a-bc18-46a7-95eb-87c56806bd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:35.635842 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dee3a-bc18-46a7-95eb-87c56806bd07 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:35.636224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dee3a-bc18-46a7-95eb-87c56806bd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:35.636507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dee3a-bc18-46a7-95eb-87c56806bd07 None admin] PATH_INFO: `/v3/users/72ac367241b34a4bafbea8748bd53095` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:35.637865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999dee3a-bc18-46a7-95eb-87c56806bd07 None admin] RBAC: Authorizing `identity:get_user(user_id=72ac367241b34a4bafbea8748bd53095)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:35.639417 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999dee3a-bc18-46a7-95eb-87c56806bd07 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:35.641196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 68/131] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:35 2022] GET /identity/v3/users/72ac367241b34a4bafbea8748bd53095 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:35.642091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dee3a-bc18-46a7-95eb-87c56806bd07 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:35.674575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 64/132] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:29:35 2022] PUT /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/users/264c9bf15b9849a18b942d5d77559b9b/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:29:35.676423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316c00d-d2ba-48ff-b509-c69ed4002076 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:35.729276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:35.787676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:35.819503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999dee3a-bc18-46a7-95eb-87c56806bd07 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:35.859478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5705a80-7052-4844-9cc3-902809ce12bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:35.860582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5705a80-7052-4844-9cc3-902809ce12bf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:35.860872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5705a80-7052-4844-9cc3-902809ce12bf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:35.861182 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5705a80-7052-4844-9cc3-902809ce12bf None admin] PATH_INFO: `/v3/users/72ac367241b34a4bafbea8748bd53095` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:35.862462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5705a80-7052-4844-9cc3-902809ce12bf None admin] RBAC: Authorizing `identity:get_user(user_id=72ac367241b34a4bafbea8748bd53095)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:35.863982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5705a80-7052-4844-9cc3-902809ce12bf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:35.865768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 69/133] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:35 2022] GET /identity/v3/users/72ac367241b34a4bafbea8748bd53095 => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:35.872868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5705a80-7052-4844-9cc3-902809ce12bf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:35.953764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316c00d-d2ba-48ff-b509-c69ed4002076 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:36.032781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26bae3-c271-44bc-b717-240f593af2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:36.033694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26bae3-c271-44bc-b717-240f593af2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:36.033984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26bae3-c271-44bc-b717-240f593af2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:36.034269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26bae3-c271-44bc-b717-240f593af2a3 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:36.040998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26bae3-c271-44bc-b717-240f593af2a3 None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:36.042404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26bae3-c271-44bc-b717-240f593af2a3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:36.050410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 65/134] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:35 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:36.060652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5705a80-7052-4844-9cc3-902809ce12bf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:36.094373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26bae3-c271-44bc-b717-240f593af2a3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:36.121664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77eefd-00d1-4ced-b490-98ed14fc72bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:36.122543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77eefd-00d1-4ced-b490-98ed14fc72bb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:36.122832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77eefd-00d1-4ced-b490-98ed14fc72bb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:36.123113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77eefd-00d1-4ced-b490-98ed14fc72bb None admin] PATH_INFO: `/v3/projects/77434394fa334e2cb6914227f2c2d09d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:36.127348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77eefd-00d1-4ced-b490-98ed14fc72bb None admin] RBAC: Authorizing `identity:get_project(project_id=77434394fa334e2cb6914227f2c2d09d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:36.137241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77eefd-00d1-4ced-b490-98ed14fc72bb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:36.139261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 70/135] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:35 2022] GET /identity/v3/projects/77434394fa334e2cb6914227f2c2d09d => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:36.149866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77eefd-00d1-4ced-b490-98ed14fc72bb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:36.186092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:36.246782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:36.262753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26bae3-c271-44bc-b717-240f593af2a3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:36.355579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77eefd-00d1-4ced-b490-98ed14fc72bb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:36.356362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba044e-8764-43f2-9947-556d3f019808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:36.357311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba044e-8764-43f2-9947-556d3f019808 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:36.357603 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba044e-8764-43f2-9947-556d3f019808 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:36.357888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba044e-8764-43f2-9947-556d3f019808 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/users/72ac367241b34a4bafbea8748bd53095/roles/c465d586f0ec4b59815b23087d757fd2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:36.360225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba044e-8764-43f2-9947-556d3f019808 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c6a41d535fe491c81d7c8ea25c933bf, user_id=72ac367241b34a4bafbea8748bd53095, role_id=c465d586f0ec4b59815b23087d757fd2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:36.361732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba044e-8764-43f2-9947-556d3f019808 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:36.415715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbaf098-1a75-42b2-ac67-f96bf3e19620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:36.416624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbaf098-1a75-42b2-ac67-f96bf3e19620 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:36.416929 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbaf098-1a75-42b2-ac67-f96bf3e19620 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:36.417213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbaf098-1a75-42b2-ac67-f96bf3e19620 None admin] PATH_INFO: `/v3/projects/77434394fa334e2cb6914227f2c2d09d/users/72ac367241b34a4bafbea8748bd53095/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:36.423050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbaf098-1a75-42b2-ac67-f96bf3e19620 None admin] RBAC: Authorizing `identity:create_grant(project_id=77434394fa334e2cb6914227f2c2d09d, user_id=72ac367241b34a4bafbea8748bd53095, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:36.424532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbaf098-1a75-42b2-ac67-f96bf3e19620 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:36.483960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 66/136] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:29:36 2022] PUT /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/users/72ac367241b34a4bafbea8748bd53095/roles/c465d586f0ec4b59815b23087d757fd2 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:29:36.485391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27881614-d551-4154-aa74-b140881afcbc None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:36.485736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27881614-d551-4154-aa74-b140881afcbc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:36.486114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27881614-d551-4154-aa74-b140881afcbc None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:36.487963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 67/137] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:36.494136 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ee642-16c8-405c-bd38-493f1ede7ce9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:36.494479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ee642-16c8-405c-bd38-493f1ede7ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:36.494767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ee642-16c8-405c-bd38-493f1ede7ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:36.515580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-aa5ee642-16c8-405c-bd38-493f1ede7ce9 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:36.522356 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aa5ee642-16c8-405c-bd38-493f1ede7ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:36.558327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 71/138] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:29:36 2022] PUT /identity/v3/projects/77434394fa334e2cb6914227f2c2d09d/users/72ac367241b34a4bafbea8748bd53095/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:29:36.623072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 68/139] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:36.667935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbaf098-1a75-42b2-ac67-f96bf3e19620 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:36.768294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:36.809583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbaf098-1a75-42b2-ac67-f96bf3e19620 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:36.841739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0ec94-c36a-4e49-9335-f8ba16d70d10 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:36.841739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0ec94-c36a-4e49-9335-f8ba16d70d10 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:36.842359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0ec94-c36a-4e49-9335-f8ba16d70d10 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:36.843972 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 69/140] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:36.853462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c8a46-26fa-43b9-a0ec-db5f4881e191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:36.854630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8a46-26fa-43b9-a0ec-db5f4881e191 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:36.876277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8a46-26fa-43b9-a0ec-db5f4881e191 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:36.876277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8a46-26fa-43b9-a0ec-db5f4881e191 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:36.876277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c8a46-26fa-43b9-a0ec-db5f4881e191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:36.876277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c8a46-26fa-43b9-a0ec-db5f4881e191 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:36.876277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 72/141] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:36.876661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0ec94-c36a-4e49-9335-f8ba16d70d10 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:36.876746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3d4dd-38b0-4794-ab93-49baed580bcf None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:36.876746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3d4dd-38b0-4794-ab93-49baed580bcf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:36.876746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3d4dd-38b0-4794-ab93-49baed580bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:36.892033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a7b3d4dd-38b0-4794-ab93-49baed580bcf None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:36.893977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a7b3d4dd-38b0-4794-ab93-49baed580bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:36.995234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 73/142] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:37.016734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:37.068591 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3d4dd-38b0-4794-ab93-49baed580bcf None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:37.086968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0ec94-c36a-4e49-9335-f8ba16d70d10 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:37.125009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c0c24-6717-4c9d-8d8b-b1cdc31cb941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:37.125706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c0c24-6717-4c9d-8d8b-b1cdc31cb941 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:37.125993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c0c24-6717-4c9d-8d8b-b1cdc31cb941 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:37.126780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c0c24-6717-4c9d-8d8b-b1cdc31cb941 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:37.126971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c0c24-6717-4c9d-8d8b-b1cdc31cb941 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:37.128523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c0c24-6717-4c9d-8d8b-b1cdc31cb941 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:37.223112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:37.271132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3d4dd-38b0-4794-ab93-49baed580bcf None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:37.299712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 70/143] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:29:36 2022] POST /identity/v3/users => generated 283 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:29:37.322589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c70e3e-d6cd-4627-884b-fe9babe93cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:37.323400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70e3e-d6cd-4627-884b-fe9babe93cad None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:37.323682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70e3e-d6cd-4627-884b-fe9babe93cad None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:37.323955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70e3e-d6cd-4627-884b-fe9babe93cad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:37.344870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c70e3e-d6cd-4627-884b-fe9babe93cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d0c70e3e-d6cd-4627-884b-fe9babe93cad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:37.353399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:37.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:37.355489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 74/144] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:37.363503 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c0c24-6717-4c9d-8d8b-b1cdc31cb941 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:37.521674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c0c24-6717-4c9d-8d8b-b1cdc31cb941 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:37.564289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:37.571442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea13792-53f4-4cb4-b37d-f03d3417a3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:37.579714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea13792-53f4-4cb4-b37d-f03d3417a3cd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:37.579714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea13792-53f4-4cb4-b37d-f03d3417a3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:37.579714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea13792-53f4-4cb4-b37d-f03d3417a3cd None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:37.579714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea13792-53f4-4cb4-b37d-f03d3417a3cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:37.579714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea13792-53f4-4cb4-b37d-f03d3417a3cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:37.579714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea13792-53f4-4cb4-b37d-f03d3417a3cd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:37.586240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 71/145] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:37.592582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c70e3e-d6cd-4627-884b-fe9babe93cad None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:37.711792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c70e3e-d6cd-4627-884b-fe9babe93cad None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:37.741825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:37.751013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a96db1-bb7b-4268-be72-6fed2cc7165d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:37.753039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a96db1-bb7b-4268-be72-6fed2cc7165d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:37.753039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a96db1-bb7b-4268-be72-6fed2cc7165d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:37.753039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a96db1-bb7b-4268-be72-6fed2cc7165d None admin] PATH_INFO: `/v3/users/9dc916d276d640d7bbc8243bd1539ee1` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:37.754679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a96db1-bb7b-4268-be72-6fed2cc7165d None admin] RBAC: Authorizing `identity:get_user(user_id=9dc916d276d640d7bbc8243bd1539ee1)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:37.756354 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a96db1-bb7b-4268-be72-6fed2cc7165d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:37.758268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 75/146] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:37 2022] GET /identity/v3/users/9dc916d276d640d7bbc8243bd1539ee1 => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:37.764169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea13792-53f4-4cb4-b37d-f03d3417a3cd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:37.876866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea13792-53f4-4cb4-b37d-f03d3417a3cd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:37.914423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f964829-6ade-46b7-a470-0f8a168258f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:37.915716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f964829-6ade-46b7-a470-0f8a168258f4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:37.916197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f964829-6ade-46b7-a470-0f8a168258f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:37.916668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f964829-6ade-46b7-a470-0f8a168258f4 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:37.918204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f964829-6ade-46b7-a470-0f8a168258f4 None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:37.919902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f964829-6ade-46b7-a470-0f8a168258f4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:37.931364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 72/147] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:37 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:37.934384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a96db1-bb7b-4268-be72-6fed2cc7165d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:38.009717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe9591-0d38-4507-9f1c-e6d7abee3a5d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:38.010046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe9591-0d38-4507-9f1c-e6d7abee3a5d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:38.010286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe9591-0d38-4507-9f1c-e6d7abee3a5d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:38.012140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 73/148] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:38.017916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd371ad-adc8-448a-bfc6-e3fde0a832e8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:38.018215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd371ad-adc8-448a-bfc6-e3fde0a832e8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:38.018497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd371ad-adc8-448a-bfc6-e3fde0a832e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:38.047065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-cdd371ad-adc8-448a-bfc6-e3fde0a832e8 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:38.048001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cdd371ad-adc8-448a-bfc6-e3fde0a832e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:38.068199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a96db1-bb7b-4268-be72-6fed2cc7165d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:38.108352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4662a5-0345-472b-8352-6ef38c52b6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:38.109826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4662a5-0345-472b-8352-6ef38c52b6a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:38.110309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4662a5-0345-472b-8352-6ef38c52b6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:38.110802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4662a5-0345-472b-8352-6ef38c52b6a9 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/users/9dc916d276d640d7bbc8243bd1539ee1/roles/4f99bdbe513b4500b96f9c530dac52ed` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:38.113047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4662a5-0345-472b-8352-6ef38c52b6a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c6a41d535fe491c81d7c8ea25c933bf, user_id=9dc916d276d640d7bbc8243bd1539ee1, role_id=4f99bdbe513b4500b96f9c530dac52ed)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:38.114853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4662a5-0345-472b-8352-6ef38c52b6a9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:38.142144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:38.156586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 74/149] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:38.180548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd371ad-adc8-448a-bfc6-e3fde0a832e8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:38.210017 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 76/150] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:29:37 2022] PUT /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/users/9dc916d276d640d7bbc8243bd1539ee1/roles/4f99bdbe513b4500b96f9c530dac52ed => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:29:38.300531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd371ad-adc8-448a-bfc6-e3fde0a832e8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:38.339857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b40ab-5492-4155-b564-5dd9fa42411a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:38.340926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b40ab-5492-4155-b564-5dd9fa42411a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:38.341382 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b40ab-5492-4155-b564-5dd9fa42411a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:38.341794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b40ab-5492-4155-b564-5dd9fa42411a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:38.358875 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b40ab-5492-4155-b564-5dd9fa42411a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f93b40ab-5492-4155-b564-5dd9fa42411a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:38.366926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:38.367967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:38.369195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:38.371218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 75/151] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:38.377586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4662a5-0345-472b-8352-6ef38c52b6a9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:38.462431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:38.526972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4662a5-0345-472b-8352-6ef38c52b6a9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:38.579108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c35bc5-59f5-4644-925c-da0c9d29c842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:38.579797 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c35bc5-59f5-4644-925c-da0c9d29c842 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:38.580083 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c35bc5-59f5-4644-925c-da0c9d29c842 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:38.580361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c35bc5-59f5-4644-925c-da0c9d29c842 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:38.581159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c35bc5-59f5-4644-925c-da0c9d29c842 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:38.581469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c35bc5-59f5-4644-925c-da0c9d29c842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:38.582879 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c35bc5-59f5-4644-925c-da0c9d29c842 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:38.594530 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 77/152] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:38.600750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b40ab-5492-4155-b564-5dd9fa42411a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:38.664410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:38.729969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b40ab-5492-4155-b564-5dd9fa42411a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:38.776380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623734e-900a-49b3-bf94-b6c4ae5912b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:38.777292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623734e-900a-49b3-bf94-b6c4ae5912b9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:38.777579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623734e-900a-49b3-bf94-b6c4ae5912b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:38.777898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623734e-900a-49b3-bf94-b6c4ae5912b9 None admin] PATH_INFO: `/v3/users/72ac367241b34a4bafbea8748bd53095` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:38.779131 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623734e-900a-49b3-bf94-b6c4ae5912b9 None admin] RBAC: Authorizing `identity:get_user(user_id=72ac367241b34a4bafbea8748bd53095)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:38.781162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623734e-900a-49b3-bf94-b6c4ae5912b9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:38.783083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 76/153] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:38 2022] GET /identity/v3/users/72ac367241b34a4bafbea8748bd53095 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:38.788800 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c35bc5-59f5-4644-925c-da0c9d29c842 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:38.814980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2420aa-de0e-4d2d-ae26-a23b5735d37d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:38.815378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2420aa-de0e-4d2d-ae26-a23b5735d37d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:38.815668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2420aa-de0e-4d2d-ae26-a23b5735d37d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:38.817404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 77/154] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:38.824922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ccdc0-9cd1-4a28-875f-7e31913c5858 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:38.825225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ccdc0-9cd1-4a28-875f-7e31913c5858 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:38.825502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ccdc0-9cd1-4a28-875f-7e31913c5858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:38.852935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-134ccdc0-9cd1-4a28-875f-7e31913c5858 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:38.854170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-134ccdc0-9cd1-4a28-875f-7e31913c5858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:38.963991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 78/155] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:39.013981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c35bc5-59f5-4644-925c-da0c9d29c842 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:39.017996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ccdc0-9cd1-4a28-875f-7e31913c5858 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:39.041995 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:39.087274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd71d33-42b2-4cc5-a880-98d4468ed21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:39.088552 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd71d33-42b2-4cc5-a880-98d4468ed21e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:39.089054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd71d33-42b2-4cc5-a880-98d4468ed21e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:39.093294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd71d33-42b2-4cc5-a880-98d4468ed21e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:39.095824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:39.197058 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ccdc0-9cd1-4a28-875f-7e31913c5858 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:39.265776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c08c7-b92e-4f08-b02d-2e70cc76ce2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:39.267582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c08c7-b92e-4f08-b02d-2e70cc76ce2d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:39.268404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c08c7-b92e-4f08-b02d-2e70cc76ce2d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:39.268900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c08c7-b92e-4f08-b02d-2e70cc76ce2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:39.269003 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd71d33-42b2-4cc5-a880-98d4468ed21e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:39.270464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd71d33-42b2-4cc5-a880-98d4468ed21e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:39.289832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c08c7-b92e-4f08-b02d-2e70cc76ce2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d83c08c7-b92e-4f08-b02d-2e70cc76ce2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:39.300182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:39.301116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:39.301896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:39.304904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 79/156] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:29:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:39.314258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c08c7-b92e-4f08-b02d-2e70cc76ce2d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:39.461572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 78/157] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:39.468365 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd71d33-42b2-4cc5-a880-98d4468ed21e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:39.489034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:39.519143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c08c7-b92e-4f08-b02d-2e70cc76ce2d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:39.532012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:39.580340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd71d33-42b2-4cc5-a880-98d4468ed21e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:39.601615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a25c0-2256-4610-86b6-8096f02955c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:39.602310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a25c0-2256-4610-86b6-8096f02955c5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:39.602592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a25c0-2256-4610-86b6-8096f02955c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:39.602867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a25c0-2256-4610-86b6-8096f02955c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:39.604010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a25c0-2256-4610-86b6-8096f02955c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:39.604010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a25c0-2256-4610-86b6-8096f02955c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:39.605407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a25c0-2256-4610-86b6-8096f02955c5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:39.619961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699307f0-2d30-4e0e-8f23-3e57966731dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:39.620403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 80/158] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:39.621557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699307f0-2d30-4e0e-8f23-3e57966731dc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:39.622038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699307f0-2d30-4e0e-8f23-3e57966731dc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:39.622508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699307f0-2d30-4e0e-8f23-3e57966731dc None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:39.624143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699307f0-2d30-4e0e-8f23-3e57966731dc None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:39.625901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a25c0-2256-4610-86b6-8096f02955c5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:39.626231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699307f0-2d30-4e0e-8f23-3e57966731dc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:39.628219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 79/159] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:39 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:39.634376 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699307f0-2d30-4e0e-8f23-3e57966731dc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:39.765415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:39.776215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699307f0-2d30-4e0e-8f23-3e57966731dc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:39.816734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a25c0-2256-4610-86b6-8096f02955c5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:39.833475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb314a21-aa0a-4aa9-ba55-032613e63a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:39.834156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb314a21-aa0a-4aa9-ba55-032613e63a16 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:39.834539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb314a21-aa0a-4aa9-ba55-032613e63a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:39.834822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb314a21-aa0a-4aa9-ba55-032613e63a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:39.835781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb314a21-aa0a-4aa9-ba55-032613e63a16 None admin] RBAC: Adding query filter params (role.id=3b2e1fad42f441e9a61c142477a50b11, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf, user.id=72ac367241b34a4bafbea8748bd53095) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:39.836183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb314a21-aa0a-4aa9-ba55-032613e63a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:39.837692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb314a21-aa0a-4aa9-ba55-032613e63a16 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:39.850413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 80/160] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:39 2022] GET /identity/v3/role_assignments?user.id=72ac367241b34a4bafbea8748bd53095&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:39.878684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51ba12-6a4c-458e-8e3f-0fc3059473f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:39.879593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51ba12-6a4c-458e-8e3f-0fc3059473f5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:39.879878 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51ba12-6a4c-458e-8e3f-0fc3059473f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:39.880156 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51ba12-6a4c-458e-8e3f-0fc3059473f5 None admin] PATH_INFO: `/v3/users/264c9bf15b9849a18b942d5d77559b9b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:39.885607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51ba12-6a4c-458e-8e3f-0fc3059473f5 None admin] RBAC: Authorizing `identity:get_user(user_id=264c9bf15b9849a18b942d5d77559b9b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:39.887082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51ba12-6a4c-458e-8e3f-0fc3059473f5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:39.889006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 81/161] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:39 2022] GET /identity/v3/users/264c9bf15b9849a18b942d5d77559b9b => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:39.894682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb314a21-aa0a-4aa9-ba55-032613e63a16 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:40.047636 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb314a21-aa0a-4aa9-ba55-032613e63a16 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:40.085962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f895f70-764a-46f0-af77-48624ed63b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:40.086627 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f895f70-764a-46f0-af77-48624ed63b08 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:40.086912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f895f70-764a-46f0-af77-48624ed63b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:40.087188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f895f70-764a-46f0-af77-48624ed63b08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:40.093319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3d6e1-ef0e-43a7-8473-311d229a3ed9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:40.093837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3d6e1-ef0e-43a7-8473-311d229a3ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:40.093837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3d6e1-ef0e-43a7-8473-311d229a3ed9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:40.095468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 82/162] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:40.102617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade41d9-cc38-40ac-a3dc-83e6c2595add None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:40.102905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade41d9-cc38-40ac-a3dc-83e6c2595add None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:40.103185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade41d9-cc38-40ac-a3dc-83e6c2595add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:40.118491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:40.148862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6ade41d9-cc38-40ac-a3dc-83e6c2595add None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:40.150110 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6ade41d9-cc38-40ac-a3dc-83e6c2595add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:40.163357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f895f70-764a-46f0-af77-48624ed63b08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:40.165007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f895f70-764a-46f0-af77-48624ed63b08 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:40.235980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:40.276738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 81/163] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:40.283283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f895f70-764a-46f0-af77-48624ed63b08 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:40.331392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 83/164] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:40.365375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:40.374794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade41d9-cc38-40ac-a3dc-83e6c2595add None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:40.429313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f895f70-764a-46f0-af77-48624ed63b08 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:40.468967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffa376-d3dd-4025-a739-9ffd6ba9c1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:40.469845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffa376-d3dd-4025-a739-9ffd6ba9c1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:40.470129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffa376-d3dd-4025-a739-9ffd6ba9c1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:40.470407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffa376-d3dd-4025-a739-9ffd6ba9c1a0 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:40.471844 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffa376-d3dd-4025-a739-9ffd6ba9c1a0 None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:40.473531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffa376-d3dd-4025-a739-9ffd6ba9c1a0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:40.475474 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 82/165] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:40 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:40.476951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bdb4a-f6ac-4f3a-a2ca-2030f62c9e5a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:40.477245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bdb4a-f6ac-4f3a-a2ca-2030f62c9e5a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:40.477520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bdb4a-f6ac-4f3a-a2ca-2030f62c9e5a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:40.479034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 83/166] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:40.481985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bdb4a-f6ac-4f3a-a2ca-2030f62c9e5a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:40.566867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade41d9-cc38-40ac-a3dc-83e6c2595add None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:40.604992 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bdb4a-f6ac-4f3a-a2ca-2030f62c9e5a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:40.623534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:40.634932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65a41f-7c1f-4ed0-8cfe-e141516b6fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:40.636072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65a41f-7c1f-4ed0-8cfe-e141516b6fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:40.636555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65a41f-7c1f-4ed0-8cfe-e141516b6fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:40.637056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65a41f-7c1f-4ed0-8cfe-e141516b6fe6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:40.647584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbad99-2dac-465b-90a4-97d6ec6f4455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:40.649359 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbad99-2dac-465b-90a4-97d6ec6f4455 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:40.649649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbad99-2dac-465b-90a4-97d6ec6f4455 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:40.650132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbad99-2dac-465b-90a4-97d6ec6f4455 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:40.652121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbad99-2dac-465b-90a4-97d6ec6f4455 None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf, user.id=264c9bf15b9849a18b942d5d77559b9b) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:40.652442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbad99-2dac-465b-90a4-97d6ec6f4455 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:40.657913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbad99-2dac-465b-90a4-97d6ec6f4455 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:40.660527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65a41f-7c1f-4ed0-8cfe-e141516b6fe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:40.668747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 84/167] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:40 2022] GET /identity/v3/role_assignments?user.id=264c9bf15b9849a18b942d5d77559b9b&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=81457691f3a2456fa33358cb6a922d05 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:40.669830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a67e0-20b1-4d66-926c-700b28fdc644 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:40.670120 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a67e0-20b1-4d66-926c-700b28fdc644 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:40.675427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a67e0-20b1-4d66-926c-700b28fdc644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-bc65a41f-7c1f-4ed0-8cfe-e141516b6fe6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:40.677837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:40.678798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:40.680073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 84/168] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:40.690134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65a41f-7c1f-4ed0-8cfe-e141516b6fe6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:40.697720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-945a67e0-20b1-4d66-926c-700b28fdc644 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:40.698944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-945a67e0-20b1-4d66-926c-700b28fdc644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:40.736290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:40.830677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 85/169] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:40.849675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65a41f-7c1f-4ed0-8cfe-e141516b6fe6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:40.869548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a67e0-20b1-4d66-926c-700b28fdc644 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:40.901533 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db209b09-aa00-4d1f-ac57-3136a5a4362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:40.902618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db209b09-aa00-4d1f-ac57-3136a5a4362b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:40.902908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db209b09-aa00-4d1f-ac57-3136a5a4362b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:40.903190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db209b09-aa00-4d1f-ac57-3136a5a4362b None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:40.903966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db209b09-aa00-4d1f-ac57-3136a5a4362b None admin] RBAC: Adding query filter params (name=member) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:40.904275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db209b09-aa00-4d1f-ac57-3136a5a4362b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:40.906338 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db209b09-aa00-4d1f-ac57-3136a5a4362b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:40.927951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 85/170] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:40.948922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db209b09-aa00-4d1f-ac57-3136a5a4362b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:40.999500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a67e0-20b1-4d66-926c-700b28fdc644 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:41.013550 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:41.037597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181cfd3-1b06-41f5-a0da-860ee4548c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:41.038429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cfd3-1b06-41f5-a0da-860ee4548c61 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:41.038705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cfd3-1b06-41f5-a0da-860ee4548c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:41.038974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cfd3-1b06-41f5-a0da-860ee4548c61 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:41.052165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181cfd3-1b06-41f5-a0da-860ee4548c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5181cfd3-1b06-41f5-a0da-860ee4548c61 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:41.060445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:41.061893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:41.063193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 86/171] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:29:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:41.068264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181cfd3-1b06-41f5-a0da-860ee4548c61 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:41.081541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:41.093015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db209b09-aa00-4d1f-ac57-3136a5a4362b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:41.132236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf40cb4-f537-4fbc-a116-4948bbdfc1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:41.133214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf40cb4-f537-4fbc-a116-4948bbdfc1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:41.133570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf40cb4-f537-4fbc-a116-4948bbdfc1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:41.133947 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf40cb4-f537-4fbc-a116-4948bbdfc1d9 None admin] PATH_INFO: `/v3/users/72ac367241b34a4bafbea8748bd53095` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:41.135254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf40cb4-f537-4fbc-a116-4948bbdfc1d9 None admin] RBAC: Authorizing `identity:get_user(user_id=72ac367241b34a4bafbea8748bd53095)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:41.136819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf40cb4-f537-4fbc-a116-4948bbdfc1d9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:41.139063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 86/172] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:40 2022] GET /identity/v3/users/72ac367241b34a4bafbea8748bd53095 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:41.145534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf40cb4-f537-4fbc-a116-4948bbdfc1d9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:41.210293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181cfd3-1b06-41f5-a0da-860ee4548c61 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:41.282589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c9cbb-0fc3-45fc-8f82-8b795d04f075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:41.283261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9cbb-0fc3-45fc-8f82-8b795d04f075 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:41.283540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9cbb-0fc3-45fc-8f82-8b795d04f075 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:41.283819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9cbb-0fc3-45fc-8f82-8b795d04f075 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:41.284581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c9cbb-0fc3-45fc-8f82-8b795d04f075 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:41.284904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c9cbb-0fc3-45fc-8f82-8b795d04f075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:41.286290 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c9cbb-0fc3-45fc-8f82-8b795d04f075 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:41.309703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 87/173] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:29:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:41.318383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c9cbb-0fc3-45fc-8f82-8b795d04f075 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:41.368341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf40cb4-f537-4fbc-a116-4948bbdfc1d9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:41.393330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:41.470168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23bd682-959b-4e7b-87cd-96df840af459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:41.470847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bd682-959b-4e7b-87cd-96df840af459 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:41.471139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bd682-959b-4e7b-87cd-96df840af459 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:41.471422 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23bd682-959b-4e7b-87cd-96df840af459 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:41.471693 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c9cbb-0fc3-45fc-8f82-8b795d04f075 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:41.513092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de226a4f-5a65-4b69-a9be-03a9ec00d984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:41.514856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de226a4f-5a65-4b69-a9be-03a9ec00d984 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:41.515281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de226a4f-5a65-4b69-a9be-03a9ec00d984 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:41.515795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de226a4f-5a65-4b69-a9be-03a9ec00d984 None admin] PATH_INFO: `/v3/users/72ac367241b34a4bafbea8748bd53095` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:41.518457 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de226a4f-5a65-4b69-a9be-03a9ec00d984 None admin] RBAC: Authorizing `identity:get_user(user_id=72ac367241b34a4bafbea8748bd53095)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:41.520132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de226a4f-5a65-4b69-a9be-03a9ec00d984 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:41.522107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 88/174] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:41 2022] GET /identity/v3/users/72ac367241b34a4bafbea8748bd53095 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:41.527663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de226a4f-5a65-4b69-a9be-03a9ec00d984 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:41.531999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:41.591041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23bd682-959b-4e7b-87cd-96df840af459 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:41.593094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23bd682-959b-4e7b-87cd-96df840af459 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:41.634968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de226a4f-5a65-4b69-a9be-03a9ec00d984 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:41.636848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:41.673069 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff68f5a-7f56-4ab3-b4e2-2179f5642017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:41.673673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68f5a-7f56-4ab3-b4e2-2179f5642017 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:41.673957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68f5a-7f56-4ab3-b4e2-2179f5642017 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:41.674229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff68f5a-7f56-4ab3-b4e2-2179f5642017 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:41.777244 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 87/175] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:41.782801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23bd682-959b-4e7b-87cd-96df840af459 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:41.814985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff68f5a-7f56-4ab3-b4e2-2179f5642017 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:41.816467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff68f5a-7f56-4ab3-b4e2-2179f5642017 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:41.900688 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:41.954864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23bd682-959b-4e7b-87cd-96df840af459 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:42.024310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bce64-8d27-4826-84e0-9acbea6ff069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:42.025225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bce64-8d27-4826-84e0-9acbea6ff069 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.025509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bce64-8d27-4826-84e0-9acbea6ff069 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.026296 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bce64-8d27-4826-84e0-9acbea6ff069 None admin] PATH_INFO: `/v3/projects/77434394fa334e2cb6914227f2c2d09d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.027716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bce64-8d27-4826-84e0-9acbea6ff069 None admin] RBAC: Authorizing `identity:get_project(project_id=77434394fa334e2cb6914227f2c2d09d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:42.029208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bce64-8d27-4826-84e0-9acbea6ff069 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:42.031497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 88/176] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:41 2022] GET /identity/v3/projects/77434394fa334e2cb6914227f2c2d09d => generated 312 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:42.035895 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 89/177] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:29:42.038415 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bce64-8d27-4826-84e0-9acbea6ff069 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:42.042897 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff68f5a-7f56-4ab3-b4e2-2179f5642017 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:42.045328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:42.236463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bce64-8d27-4826-84e0-9acbea6ff069 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:42.249656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff68f5a-7f56-4ab3-b4e2-2179f5642017 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:42.293783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:42.308081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87f733-b136-4020-a607-7d52921ab823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:42.311013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87f733-b136-4020-a607-7d52921ab823 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.311530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87f733-b136-4020-a607-7d52921ab823 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.311960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87f733-b136-4020-a607-7d52921ab823 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.312998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87f733-b136-4020-a607-7d52921ab823 None admin] RBAC: Adding query filter params (role.id=3b2e1fad42f441e9a61c142477a50b11, scope.project.id=77434394fa334e2cb6914227f2c2d09d, user.id=72ac367241b34a4bafbea8748bd53095) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:42.313476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87f733-b136-4020-a607-7d52921ab823 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:42.315187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87f733-b136-4020-a607-7d52921ab823 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:42.328458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 89/178] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:42 2022] GET /identity/v3/role_assignments?user.id=72ac367241b34a4bafbea8748bd53095&scope.project.id=77434394fa334e2cb6914227f2c2d09d&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:42.334567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f169719-9cda-40ce-86c1-3fb6fa7cf3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:42.335467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f169719-9cda-40ce-86c1-3fb6fa7cf3ba None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.335749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f169719-9cda-40ce-86c1-3fb6fa7cf3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.336026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f169719-9cda-40ce-86c1-3fb6fa7cf3ba None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.341922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f169719-9cda-40ce-86c1-3fb6fa7cf3ba None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:42.344004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f169719-9cda-40ce-86c1-3fb6fa7cf3ba None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:42.345863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 90/179] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:42 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:42.351856 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87f733-b136-4020-a607-7d52921ab823 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:42.423660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c92c8-04a8-48f6-9708-4d772df4275b None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.424679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c92c8-04a8-48f6-9708-4d772df4275b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.424679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c92c8-04a8-48f6-9708-4d772df4275b None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.425781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 91/180] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:29:42.431257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14464-fb8f-4d62-8c39-6e5374d13f43 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.431673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14464-fb8f-4d62-8c39-6e5374d13f43 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.432070 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14464-fb8f-4d62-8c39-6e5374d13f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.470093 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e3e14464-fb8f-4d62-8c39-6e5374d13f43 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:42.471761 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e3e14464-fb8f-4d62-8c39-6e5374d13f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:42.566930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87f733-b136-4020-a607-7d52921ab823 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:42.589125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:42.597917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 92/181] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:42.597917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e429eb-02e8-4705-8196-aecbe606a493 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.598197 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e429eb-02e8-4705-8196-aecbe606a493 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.598360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e429eb-02e8-4705-8196-aecbe606a493 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.599944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 93/182] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:42.601223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5d925-c19e-4b65-bdb3-1e266d8a3e13 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.608953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5d925-c19e-4b65-bdb3-1e266d8a3e13 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.608953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5d925-c19e-4b65-bdb3-1e266d8a3e13 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.608953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 94/183] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:42.608953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7a2c7-8e07-4144-ad50-52e5b47028d0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.608953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7a2c7-8e07-4144-ad50-52e5b47028d0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.608953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7a2c7-8e07-4144-ad50-52e5b47028d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.621077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20c7a4-ca80-47fe-94aa-20748898416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:42.622115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20c7a4-ca80-47fe-94aa-20748898416c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.622645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20c7a4-ca80-47fe-94aa-20748898416c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.623103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20c7a4-ca80-47fe-94aa-20748898416c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.624184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20c7a4-ca80-47fe-94aa-20748898416c None admin] RBAC: Adding query filter params (role.id=c465d586f0ec4b59815b23087d757fd2, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf, user.id=72ac367241b34a4bafbea8748bd53095) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:42.624576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20c7a4-ca80-47fe-94aa-20748898416c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:42.626307 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20c7a4-ca80-47fe-94aa-20748898416c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:42.628594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d4d7a2c7-8e07-4144-ad50-52e5b47028d0 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:42.630430 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d4d7a2c7-8e07-4144-ad50-52e5b47028d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:42.635413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 90/184] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:42 2022] GET /identity/v3/role_assignments?user.id=72ac367241b34a4bafbea8748bd53095&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=c465d586f0ec4b59815b23087d757fd2 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:42.636918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc080c-fd00-4aca-9e5c-30a430e95cad None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.637230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc080c-fd00-4aca-9e5c-30a430e95cad None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.637513 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc080c-fd00-4aca-9e5c-30a430e95cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.656846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8cdc080c-fd00-4aca-9e5c-30a430e95cad None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:42.658080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8cdc080c-fd00-4aca-9e5c-30a430e95cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:42.712199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 95/185] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:42.713205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7a2c7-8e07-4144-ad50-52e5b47028d0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:42.728405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:42.751467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 91/186] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:42.757231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb3a80-ee84-4c38-8109-2a3f1572c919 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.757702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb3a80-ee84-4c38-8109-2a3f1572c919 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.758029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb3a80-ee84-4c38-8109-2a3f1572c919 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.759938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 92/187] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:42.760742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb3a80-ee84-4c38-8109-2a3f1572c919 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:42.855710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7a2c7-8e07-4144-ad50-52e5b47028d0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:42.891103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:42.894554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb3a80-ee84-4c38-8109-2a3f1572c919 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:42.897270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4ed99-1eca-42b4-964a-89c87b2ec265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:42.898082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4ed99-1eca-42b4-964a-89c87b2ec265 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.898366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4ed99-1eca-42b4-964a-89c87b2ec265 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.898639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4ed99-1eca-42b4-964a-89c87b2ec265 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.909204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4ed99-1eca-42b4-964a-89c87b2ec265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f3c4ed99-1eca-42b4-964a-89c87b2ec265 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:42.916968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:42.917847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:42.918634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 96/188] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:29:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:42.919623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa036501-e12c-4126-a5d7-81a7853095df None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.919802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa036501-e12c-4126-a5d7-81a7853095df None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.920044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa036501-e12c-4126-a5d7-81a7853095df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.932769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e5185-478e-4984-85f9-6ddaac9a2188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:42.933617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e5185-478e-4984-85f9-6ddaac9a2188 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:42.933904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e5185-478e-4984-85f9-6ddaac9a2188 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:42.934183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e5185-478e-4984-85f9-6ddaac9a2188 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:42.938186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fa036501-e12c-4126-a5d7-81a7853095df None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:42.939390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fa036501-e12c-4126-a5d7-81a7853095df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:42.947233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e5185-478e-4984-85f9-6ddaac9a2188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1e4e5185-478e-4984-85f9-6ddaac9a2188 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:42.955532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:42.957460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:42.959182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 93/189] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:42.962059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e5185-478e-4984-85f9-6ddaac9a2188 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:43.009372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:43.009667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 97/190] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:43.010560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa036501-e12c-4126-a5d7-81a7853095df None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:43.067769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e5185-478e-4984-85f9-6ddaac9a2188 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:43.101849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c16d1f-ea32-4d19-8335-be4680ecc43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:43.102681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c16d1f-ea32-4d19-8335-be4680ecc43f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:43.102971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c16d1f-ea32-4d19-8335-be4680ecc43f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:43.103247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c16d1f-ea32-4d19-8335-be4680ecc43f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:43.113563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c16d1f-ea32-4d19-8335-be4680ecc43f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:43.113688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa036501-e12c-4126-a5d7-81a7853095df None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c4c16d1f-ea32-4d19-8335-be4680ecc43f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:43.121753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:43.122797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 94/191] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:29:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:43.124316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c16d1f-ea32-4d19-8335-be4680ecc43f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:43.136538 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:43.151210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eadeb6-d24b-46ff-9cf1-ba1c470fd8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:43.151885 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eadeb6-d24b-46ff-9cf1-ba1c470fd8fb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:43.152171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eadeb6-d24b-46ff-9cf1-ba1c470fd8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:43.152451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eadeb6-d24b-46ff-9cf1-ba1c470fd8fb None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:43.153261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eadeb6-d24b-46ff-9cf1-ba1c470fd8fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:43.153570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eadeb6-d24b-46ff-9cf1-ba1c470fd8fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:43.154998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eadeb6-d24b-46ff-9cf1-ba1c470fd8fb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:43.163458 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 98/192] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:43.164192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eadeb6-d24b-46ff-9cf1-ba1c470fd8fb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:43.239176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c16d1f-ea32-4d19-8335-be4680ecc43f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:43.241566 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:43.274369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaa2ef-520a-431d-9c14-2b20c429a47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:43.275051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaa2ef-520a-431d-9c14-2b20c429a47e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:43.275340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaa2ef-520a-431d-9c14-2b20c429a47e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:43.275620 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaa2ef-520a-431d-9c14-2b20c429a47e None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:43.276378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfaa2ef-520a-431d-9c14-2b20c429a47e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:43.276683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfaa2ef-520a-431d-9c14-2b20c429a47e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:43.277504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eadeb6-d24b-46ff-9cf1-ba1c470fd8fb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:43.278116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfaa2ef-520a-431d-9c14-2b20c429a47e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:43.287422 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 95/193] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:43.288278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaa2ef-520a-431d-9c14-2b20c429a47e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:43.312954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a4a40-1a1d-42a1-a628-b8cb9bc11ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:43.313744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a4a40-1a1d-42a1-a628-b8cb9bc11ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:43.314027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a4a40-1a1d-42a1-a628-b8cb9bc11ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:43.314303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a4a40-1a1d-42a1-a628-b8cb9bc11ad9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:43.315483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a4a40-1a1d-42a1-a628-b8cb9bc11ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:43.316981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a4a40-1a1d-42a1-a628-b8cb9bc11ad9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:43.318770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 99/194] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:43.320637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a4a40-1a1d-42a1-a628-b8cb9bc11ad9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:43.371859 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:43.393687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaa2ef-520a-431d-9c14-2b20c429a47e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:43.423491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a4a40-1a1d-42a1-a628-b8cb9bc11ad9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:43.431509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957d62f-f7a1-4696-8b29-248e4c0fa68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:43.432183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957d62f-f7a1-4696-8b29-248e4c0fa68f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:43.432469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957d62f-f7a1-4696-8b29-248e4c0fa68f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:43.432748 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957d62f-f7a1-4696-8b29-248e4c0fa68f None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:43.433539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957d62f-f7a1-4696-8b29-248e4c0fa68f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:43.433847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957d62f-f7a1-4696-8b29-248e4c0fa68f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:43.435237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957d62f-f7a1-4696-8b29-248e4c0fa68f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:43.443311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 96/195] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:29:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:43.445238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957d62f-f7a1-4696-8b29-248e4c0fa68f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:43.457326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24f4f9-ab3c-402b-8d64-bda6c9c6b1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:43.458193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24f4f9-ab3c-402b-8d64-bda6c9c6b1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:43.458478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24f4f9-ab3c-402b-8d64-bda6c9c6b1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:43.458749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24f4f9-ab3c-402b-8d64-bda6c9c6b1f2 None admin] PATH_INFO: `/v3/users/09e7fd26e414495691cc851df63896a7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:43.472664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24f4f9-ab3c-402b-8d64-bda6c9c6b1f2 None admin] RBAC: Authorizing `identity:get_user(user_id=09e7fd26e414495691cc851df63896a7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:43.473949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:43.474166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de24f4f9-ab3c-402b-8d64-bda6c9c6b1f2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:43.475923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 100/196] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:43 2022] GET /identity/v3/users/09e7fd26e414495691cc851df63896a7 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:43.476663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24f4f9-ab3c-402b-8d64-bda6c9c6b1f2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:43.549492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957d62f-f7a1-4696-8b29-248e4c0fa68f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:43.577857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24f4f9-ab3c-402b-8d64-bda6c9c6b1f2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:43.583194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66552147-6f9b-4510-8cff-154a6a45cac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:43.584066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66552147-6f9b-4510-8cff-154a6a45cac0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:43.584413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66552147-6f9b-4510-8cff-154a6a45cac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:43.584624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66552147-6f9b-4510-8cff-154a6a45cac0 None admin] PATH_INFO: `/v3/users/9dc916d276d640d7bbc8243bd1539ee1` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:43.585862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66552147-6f9b-4510-8cff-154a6a45cac0 None admin] RBAC: Authorizing `identity:get_user(user_id=9dc916d276d640d7bbc8243bd1539ee1)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:43.587174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:43.587349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66552147-6f9b-4510-8cff-154a6a45cac0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:43.589013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 97/197] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:43 2022] GET /identity/v3/users/9dc916d276d640d7bbc8243bd1539ee1 => generated 289 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:43.589789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66552147-6f9b-4510-8cff-154a6a45cac0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:43.613379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77a8f2-fe93-4196-84c6-01452b423f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:43.614225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a8f2-fe93-4196-84c6-01452b423f94 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:43.614519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a8f2-fe93-4196-84c6-01452b423f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:43.614816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77a8f2-fe93-4196-84c6-01452b423f94 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:43.615510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77a8f2-fe93-4196-84c6-01452b423f94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:43.616982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77a8f2-fe93-4196-84c6-01452b423f94 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:43.683581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:43.690059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66552147-6f9b-4510-8cff-154a6a45cac0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:43.706178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 101/198] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:29:43 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:29:43.715480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77a8f2-fe93-4196-84c6-01452b423f94 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:43.726423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597d2ae-f3c8-4bcc-9993-8f0a57cc3214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:43.727369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d2ae-f3c8-4bcc-9993-8f0a57cc3214 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:43.729571 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d2ae-f3c8-4bcc-9993-8f0a57cc3214 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:43.729571 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d2ae-f3c8-4bcc-9993-8f0a57cc3214 None admin] PATH_INFO: `/v3/users/09e7fd26e414495691cc851df63896a7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:43.729749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597d2ae-f3c8-4bcc-9993-8f0a57cc3214 None admin] RBAC: Authorizing `identity:get_user(user_id=09e7fd26e414495691cc851df63896a7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:43.731344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597d2ae-f3c8-4bcc-9993-8f0a57cc3214 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:43.734197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 98/199] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:43 2022] GET /identity/v3/users/09e7fd26e414495691cc851df63896a7 => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:43.735213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597d2ae-f3c8-4bcc-9993-8f0a57cc3214 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:43.861972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77a8f2-fe93-4196-84c6-01452b423f94 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:43.862183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597d2ae-f3c8-4bcc-9993-8f0a57cc3214 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:43.902968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1876afd-1483-49a9-af67-740e3612286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:43.903560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1876afd-1483-49a9-af67-740e3612286f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:43.903846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1876afd-1483-49a9-af67-740e3612286f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:43.904176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1876afd-1483-49a9-af67-740e3612286f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:43.915825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3ef4d-c9ca-472f-b1fc-6878b53c54df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:43.916476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3ef4d-c9ca-472f-b1fc-6878b53c54df None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:43.916758 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3ef4d-c9ca-472f-b1fc-6878b53c54df None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:43.917085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3ef4d-c9ca-472f-b1fc-6878b53c54df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:43.947996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:43.961134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:44.003812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1876afd-1483-49a9-af67-740e3612286f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:44.005402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1876afd-1483-49a9-af67-740e3612286f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:44.015832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3ef4d-c9ca-472f-b1fc-6878b53c54df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:44.018294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3ef4d-c9ca-472f-b1fc-6878b53c54df None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:44.133604 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 102/200] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:44.135357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1876afd-1483-49a9-af67-740e3612286f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:44.147028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 99/201] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:44.147964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3ef4d-c9ca-472f-b1fc-6878b53c54df None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:44.216288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:44.231548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:44.269843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1876afd-1483-49a9-af67-740e3612286f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:44.283903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3ef4d-c9ca-472f-b1fc-6878b53c54df None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:44.319982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619012ed-2b9c-4ee6-8b17-c7af827bd9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:44.320794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619012ed-2b9c-4ee6-8b17-c7af827bd9bb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:44.321085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619012ed-2b9c-4ee6-8b17-c7af827bd9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:44.321368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619012ed-2b9c-4ee6-8b17-c7af827bd9bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:44.326103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782cd8d-9749-4781-8ca2-f637d4bac7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:44.327555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782cd8d-9749-4781-8ca2-f637d4bac7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:44.327841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782cd8d-9749-4781-8ca2-f637d4bac7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:44.328120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782cd8d-9749-4781-8ca2-f637d4bac7d8 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:44.329388 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782cd8d-9749-4781-8ca2-f637d4bac7d8 None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:44.330869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782cd8d-9749-4781-8ca2-f637d4bac7d8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:44.332600 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 100/202] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:44 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:44.333421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782cd8d-9749-4781-8ca2-f637d4bac7d8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:44.412704 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619012ed-2b9c-4ee6-8b17-c7af827bd9bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:44.414268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619012ed-2b9c-4ee6-8b17-c7af827bd9bb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:44.443627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782cd8d-9749-4781-8ca2-f637d4bac7d8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:44.476411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:44.481865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53608801-b7e9-4976-bae7-31a80b613e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:44.482831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53608801-b7e9-4976-bae7-31a80b613e57 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:44.483214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53608801-b7e9-4976-bae7-31a80b613e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:44.483561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53608801-b7e9-4976-bae7-31a80b613e57 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:44.484944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53608801-b7e9-4976-bae7-31a80b613e57 None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:44.490828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53608801-b7e9-4976-bae7-31a80b613e57 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:44.490828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 101/203] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:44 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:44.490828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53608801-b7e9-4976-bae7-31a80b613e57 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:44.491213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:44.539079 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 103/204] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:44.540013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619012ed-2b9c-4ee6-8b17-c7af827bd9bb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:44.597780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53608801-b7e9-4976-bae7-31a80b613e57 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:44.635059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb623a5-7235-46a0-a912-e77623b01e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:44.635729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb623a5-7235-46a0-a912-e77623b01e7b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:44.636015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb623a5-7235-46a0-a912-e77623b01e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:44.636298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb623a5-7235-46a0-a912-e77623b01e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:44.637196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb623a5-7235-46a0-a912-e77623b01e7b None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=c5de3bc9e442415487283bf9414ae97d, user.id=09e7fd26e414495691cc851df63896a7) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:44.637509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb623a5-7235-46a0-a912-e77623b01e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:44.638968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb623a5-7235-46a0-a912-e77623b01e7b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:44.645110 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 102/205] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:44 2022] GET /identity/v3/role_assignments?user.id=09e7fd26e414495691cc851df63896a7&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=81457691f3a2456fa33358cb6a922d05 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:44.645995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb623a5-7235-46a0-a912-e77623b01e7b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:44.658140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619012ed-2b9c-4ee6-8b17-c7af827bd9bb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:44.714685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:44.723515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf3835-1767-4539-bbb4-7ff29de313b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:44.724196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf3835-1767-4539-bbb4-7ff29de313b9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:44.724527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf3835-1767-4539-bbb4-7ff29de313b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:44.724808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf3835-1767-4539-bbb4-7ff29de313b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:44.725722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf3835-1767-4539-bbb4-7ff29de313b9 None admin] RBAC: Adding query filter params (role.id=4f99bdbe513b4500b96f9c530dac52ed, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf, user.id=9dc916d276d640d7bbc8243bd1539ee1) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:44.731045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf3835-1767-4539-bbb4-7ff29de313b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:44.731045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf3835-1767-4539-bbb4-7ff29de313b9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:44.734320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 104/206] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:44 2022] GET /identity/v3/role_assignments?user.id=9dc916d276d640d7bbc8243bd1539ee1&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=4f99bdbe513b4500b96f9c530dac52ed => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:44.760057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb623a5-7235-46a0-a912-e77623b01e7b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:44.802080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1baed-4f1e-4b37-98b8-8c39f1539aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:44.802974 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1baed-4f1e-4b37-98b8-8c39f1539aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:44.803316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1baed-4f1e-4b37-98b8-8c39f1539aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:44.803596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1baed-4f1e-4b37-98b8-8c39f1539aa5 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:44.804918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1baed-4f1e-4b37-98b8-8c39f1539aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:44.806595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1baed-4f1e-4b37-98b8-8c39f1539aa5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:44.808804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 103/207] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:44 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:44.815227 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf3835-1767-4539-bbb4-7ff29de313b9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:44.890026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:44.971966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf3835-1767-4539-bbb4-7ff29de313b9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:45.009202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544a6eb-1d3f-4b69-974a-46a9a9f9817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:45.009878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a6eb-1d3f-4b69-974a-46a9a9f9817c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:45.010164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a6eb-1d3f-4b69-974a-46a9a9f9817c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:45.010445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a6eb-1d3f-4b69-974a-46a9a9f9817c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:45.011306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a6eb-1d3f-4b69-974a-46a9a9f9817c None admin] RBAC: Adding query filter params (role.id=c465d586f0ec4b59815b23087d757fd2, scope.project.id=c5de3bc9e442415487283bf9414ae97d, user.id=09e7fd26e414495691cc851df63896a7) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:45.011615 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a6eb-1d3f-4b69-974a-46a9a9f9817c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:45.013160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a6eb-1d3f-4b69-974a-46a9a9f9817c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:45.020525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 105/208] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:44 2022] GET /identity/v3/role_assignments?user.id=09e7fd26e414495691cc851df63896a7&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=c465d586f0ec4b59815b23087d757fd2 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:45.983530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ef8b0-6154-4f5e-a809-1f7d25c059be None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:45.984492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ef8b0-6154-4f5e-a809-1f7d25c059be None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:45.985048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ef8b0-6154-4f5e-a809-1f7d25c059be None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:45.986640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 104/209] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:45.992172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba90942-aade-4448-b03a-c012cfccdb26 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:45.993068 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba90942-aade-4448-b03a-c012cfccdb26 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:45.993840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba90942-aade-4448-b03a-c012cfccdb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:46.012681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cba90942-aade-4448-b03a-c012cfccdb26 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:46.014448 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cba90942-aade-4448-b03a-c012cfccdb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:46.050950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede2908-ec5f-4f5f-8813-313882b0ca56 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:46.051587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede2908-ec5f-4f5f-8813-313882b0ca56 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:46.052070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede2908-ec5f-4f5f-8813-313882b0ca56 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:46.054984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 105/210] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:29:46.059358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2062b-8c6f-4f9b-a2af-4d07a50ffa9c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:46.059913 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2062b-8c6f-4f9b-a2af-4d07a50ffa9c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:46.060391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2062b-8c6f-4f9b-a2af-4d07a50ffa9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:46.078124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-79c2062b-8c6f-4f9b-a2af-4d07a50ffa9c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:46.079637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-79c2062b-8c6f-4f9b-a2af-4d07a50ffa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:46.085287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 106/211] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:46.108571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba90942-aade-4448-b03a-c012cfccdb26 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:46.137339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:46.151988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 106/212] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:46.177303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2062b-8c6f-4f9b-a2af-4d07a50ffa9c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:46.213629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba90942-aade-4448-b03a-c012cfccdb26 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:46.244452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:46.254700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c67d59-fa31-4234-ac43-d96ce814437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:46.256465 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c67d59-fa31-4234-ac43-d96ce814437c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:46.256845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c67d59-fa31-4234-ac43-d96ce814437c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:46.257230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c67d59-fa31-4234-ac43-d96ce814437c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:46.271570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c67d59-fa31-4234-ac43-d96ce814437c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e6c67d59-fa31-4234-ac43-d96ce814437c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:46.282688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:46.283791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:46.284597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 107/213] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:46.291732 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c67d59-fa31-4234-ac43-d96ce814437c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:46.292682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2062b-8c6f-4f9b-a2af-4d07a50ffa9c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:46.341463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fbc6f-eefc-4ea8-9088-22d1f7443c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:46.342817 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fbc6f-eefc-4ea8-9088-22d1f7443c80 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:46.343105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fbc6f-eefc-4ea8-9088-22d1f7443c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:46.343383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fbc6f-eefc-4ea8-9088-22d1f7443c80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:46.345022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648fbc6f-eefc-4ea8-9088-22d1f7443c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:46.346807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648fbc6f-eefc-4ea8-9088-22d1f7443c80 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:46.349241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 107/214] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:46.354789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fbc6f-eefc-4ea8-9088-22d1f7443c80 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:46.393034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:46.413134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c67d59-fa31-4234-ac43-d96ce814437c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:46.452503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16af76-95b8-4f62-b648-f7288f31c578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:46.453671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16af76-95b8-4f62-b648-f7288f31c578 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:46.454201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16af76-95b8-4f62-b648-f7288f31c578 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:46.454726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16af76-95b8-4f62-b648-f7288f31c578 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:46.455741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16af76-95b8-4f62-b648-f7288f31c578 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:46.456241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16af76-95b8-4f62-b648-f7288f31c578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:46.458029 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16af76-95b8-4f62-b648-f7288f31c578 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:46.472363 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fbc6f-eefc-4ea8-9088-22d1f7443c80 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:46.472806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 108/215] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:46.474313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16af76-95b8-4f62-b648-f7288f31c578 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:46.502316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:46.508027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34c1cd-7c8d-4773-ad6a-2351d123137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:46.508704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34c1cd-7c8d-4773-ad6a-2351d123137e None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:46.509017 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34c1cd-7c8d-4773-ad6a-2351d123137e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:46.509294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34c1cd-7c8d-4773-ad6a-2351d123137e None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:46.510102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34c1cd-7c8d-4773-ad6a-2351d123137e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:46.511660 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34c1cd-7c8d-4773-ad6a-2351d123137e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:46.590474 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16af76-95b8-4f62-b648-f7288f31c578 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:46.604180 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 108/216] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:29:46 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:29:46.630409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07f805-cc77-4a0e-8090-d2f6c302a5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:46.631847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07f805-cc77-4a0e-8090-d2f6c302a5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:46.632380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07f805-cc77-4a0e-8090-d2f6c302a5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:46.632942 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07f805-cc77-4a0e-8090-d2f6c302a5c7 None admin] PATH_INFO: `/v3/users/2c209b7964d54419b4f8395278e9aff9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:46.649996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07f805-cc77-4a0e-8090-d2f6c302a5c7 None admin] RBAC: Authorizing `identity:get_user(user_id=2c209b7964d54419b4f8395278e9aff9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:46.651950 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07f805-cc77-4a0e-8090-d2f6c302a5c7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:46.658870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 109/217] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:46 2022] GET /identity/v3/users/2c209b7964d54419b4f8395278e9aff9 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:46.660639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34c1cd-7c8d-4773-ad6a-2351d123137e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:46.781568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34c1cd-7c8d-4773-ad6a-2351d123137e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:46.819933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3593547-07f1-4a89-abf7-8264782c1a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:46.821080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3593547-07f1-4a89-abf7-8264782c1a3c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:46.821580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3593547-07f1-4a89-abf7-8264782c1a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:46.822107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3593547-07f1-4a89-abf7-8264782c1a3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:46.884959 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:46.888718 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92cb69-96cb-4141-a778-ee377dd7b4d3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:46.889259 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92cb69-96cb-4141-a778-ee377dd7b4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:46.889536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92cb69-96cb-4141-a778-ee377dd7b4d3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:46.891067 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 110/218] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:29:46.896752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65911337-c61e-47ab-adad-c27cbf5c6228 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:46.897063 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65911337-c61e-47ab-adad-c27cbf5c6228 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:46.897338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65911337-c61e-47ab-adad-c27cbf5c6228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:46.901644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3593547-07f1-4a89-abf7-8264782c1a3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:46.903228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3593547-07f1-4a89-abf7-8264782c1a3c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:46.903727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:46.919480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-65911337-c61e-47ab-adad-c27cbf5c6228 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:46.920848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-65911337-c61e-47ab-adad-c27cbf5c6228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:47.025594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 111/219] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:47.034236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 109/220] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:47.039768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65911337-c61e-47ab-adad-c27cbf5c6228 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:47.053339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3593547-07f1-4a89-abf7-8264782c1a3c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:47.155955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:47.177123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65911337-c61e-47ab-adad-c27cbf5c6228 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:47.177350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3593547-07f1-4a89-abf7-8264782c1a3c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:47.215850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bfc47-847e-4233-a766-ff525c408c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:47.216908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bfc47-847e-4233-a766-ff525c408c47 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:47.217195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bfc47-847e-4233-a766-ff525c408c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:47.218002 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bfc47-847e-4233-a766-ff525c408c47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:47.221079 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:47.227083 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316d56a-ce30-4c87-9708-027e7c2692b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:47.227991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316d56a-ce30-4c87-9708-027e7c2692b6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:47.228656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316d56a-ce30-4c87-9708-027e7c2692b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:47.229429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316d56a-ce30-4c87-9708-027e7c2692b6 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:47.235732 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316d56a-ce30-4c87-9708-027e7c2692b6 None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:47.235854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bfc47-847e-4233-a766-ff525c408c47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:47.235955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316d56a-ce30-4c87-9708-027e7c2692b6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:47.235955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 112/221] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:47 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:47.238126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e22a4-0c9e-4ca3-acfc-87b96fda4c4e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:47.238410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e22a4-0c9e-4ca3-acfc-87b96fda4c4e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:47.238729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e22a4-0c9e-4ca3-acfc-87b96fda4c4e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:47.240045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 113/222] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8d2bfc47-847e-4233-a766-ff525c408c47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:47.242433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:47.243279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:47.244417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 110/223] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:29:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:47.246854 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e22a4-0c9e-4ca3-acfc-87b96fda4c4e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:47.247032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554a8d2-87da-4a56-a8ad-810be57b3bbd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:47.247032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554a8d2-87da-4a56-a8ad-810be57b3bbd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:47.247032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554a8d2-87da-4a56-a8ad-810be57b3bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:47.273133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5554a8d2-87da-4a56-a8ad-810be57b3bbd None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:47.274372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5554a8d2-87da-4a56-a8ad-810be57b3bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:47.360579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 111/224] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:47.362120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554a8d2-87da-4a56-a8ad-810be57b3bbd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:47.393897 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e22a4-0c9e-4ca3-acfc-87b96fda4c4e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:47.433446 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff71e6-93e0-400f-9d8b-61b1515514ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:47.434504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff71e6-93e0-400f-9d8b-61b1515514ed None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:47.434864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff71e6-93e0-400f-9d8b-61b1515514ed None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:47.435215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff71e6-93e0-400f-9d8b-61b1515514ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:47.436686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff71e6-93e0-400f-9d8b-61b1515514ed None admin] RBAC: Adding query filter params (role.id=3b2e1fad42f441e9a61c142477a50b11, scope.project.id=c5de3bc9e442415487283bf9414ae97d, user.id=2c209b7964d54419b4f8395278e9aff9) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:47.437252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff71e6-93e0-400f-9d8b-61b1515514ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:47.439210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff71e6-93e0-400f-9d8b-61b1515514ed None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:47.452925 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 114/225] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:47 2022] GET /identity/v3/role_assignments?user.id=2c209b7964d54419b4f8395278e9aff9&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:47.452925 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff71e6-93e0-400f-9d8b-61b1515514ed None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:47.509395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:47.541001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554a8d2-87da-4a56-a8ad-810be57b3bbd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:47.578058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:47.620986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482e90-b514-4429-b27a-c4d51c994c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:47.621603 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482e90-b514-4429-b27a-c4d51c994c17 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:47.621891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482e90-b514-4429-b27a-c4d51c994c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:47.622170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92482e90-b514-4429-b27a-c4d51c994c17 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:47.622970 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92482e90-b514-4429-b27a-c4d51c994c17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:47.624700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92482e90-b514-4429-b27a-c4d51c994c17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:47.627402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92482e90-b514-4429-b27a-c4d51c994c17 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:47.642076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff71e6-93e0-400f-9d8b-61b1515514ed None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:47.645675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 112/226] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:47.651814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482e90-b514-4429-b27a-c4d51c994c17 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:47.694222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e0010-2f8c-45e4-bd2d-ff5043367320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:47.695423 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e0010-2f8c-45e4-bd2d-ff5043367320 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:47.695910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e0010-2f8c-45e4-bd2d-ff5043367320 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:47.696386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e0010-2f8c-45e4-bd2d-ff5043367320 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:47.708813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e0010-2f8c-45e4-bd2d-ff5043367320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8c3e0010-2f8c-45e4-bd2d-ff5043367320 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:47.718460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:47.719351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:47.720126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:47.723005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 115/227] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:29:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:47.728486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e0010-2f8c-45e4-bd2d-ff5043367320 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:47.772979 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92482e90-b514-4429-b27a-c4d51c994c17 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:47.833661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1122a-d3d7-43b2-9882-d9e8e2435e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:47.834571 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1122a-d3d7-43b2-9882-d9e8e2435e0a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:47.834937 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1122a-d3d7-43b2-9882-d9e8e2435e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:47.835141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1122a-d3d7-43b2-9882-d9e8e2435e0a None admin] PATH_INFO: `/v3/users/09e7fd26e414495691cc851df63896a7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:47.836464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1122a-d3d7-43b2-9882-d9e8e2435e0a None admin] RBAC: Authorizing `identity:get_user(user_id=09e7fd26e414495691cc851df63896a7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:47.838425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1122a-d3d7-43b2-9882-d9e8e2435e0a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:47.840561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 113/228] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:47 2022] GET /identity/v3/users/09e7fd26e414495691cc851df63896a7 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:47.846302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1122a-d3d7-43b2-9882-d9e8e2435e0a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:47.847825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e0010-2f8c-45e4-bd2d-ff5043367320 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:47.857075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:47.879715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:47.896431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c102e1-43b2-4d02-8f19-9ac0d3faf253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:47.897151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c102e1-43b2-4d02-8f19-9ac0d3faf253 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:47.897439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c102e1-43b2-4d02-8f19-9ac0d3faf253 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:47.897717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c102e1-43b2-4d02-8f19-9ac0d3faf253 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:47.898489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c102e1-43b2-4d02-8f19-9ac0d3faf253 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:47.898798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c102e1-43b2-4d02-8f19-9ac0d3faf253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:47.900248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c102e1-43b2-4d02-8f19-9ac0d3faf253 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:47.932244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 116/229] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:29:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:47.941943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c102e1-43b2-4d02-8f19-9ac0d3faf253 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:47.975014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1122a-d3d7-43b2-9882-d9e8e2435e0a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:48.013266 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6429141-621d-4f70-bd9e-7cf0a0c960de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:48.014648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429141-621d-4f70-bd9e-7cf0a0c960de None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:48.015201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429141-621d-4f70-bd9e-7cf0a0c960de None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:48.015627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6429141-621d-4f70-bd9e-7cf0a0c960de None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:48.017009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6429141-621d-4f70-bd9e-7cf0a0c960de None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:48.018623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6429141-621d-4f70-bd9e-7cf0a0c960de None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:48.020915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 114/230] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:47 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:48.025981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6429141-621d-4f70-bd9e-7cf0a0c960de None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:48.113679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:48.137830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6429141-621d-4f70-bd9e-7cf0a0c960de None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:48.176966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c102e1-43b2-4d02-8f19-9ac0d3faf253 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:48.178265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76985ce2-557f-4410-901a-e5b397c0d00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:48.179400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76985ce2-557f-4410-901a-e5b397c0d00a None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:48.180682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76985ce2-557f-4410-901a-e5b397c0d00a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:48.181183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76985ce2-557f-4410-901a-e5b397c0d00a None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d/users/09e7fd26e414495691cc851df63896a7/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:48.186708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76985ce2-557f-4410-901a-e5b397c0d00a None admin] RBAC: Authorizing `identity:create_grant(project_id=c5de3bc9e442415487283bf9414ae97d, user_id=09e7fd26e414495691cc851df63896a7, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:48.191370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76985ce2-557f-4410-901a-e5b397c0d00a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:48.226240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dd3a4-344d-4313-a5b3-b1b3940af330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:48.227603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dd3a4-344d-4313-a5b3-b1b3940af330 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:48.228092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dd3a4-344d-4313-a5b3-b1b3940af330 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:48.228620 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9dd3a4-344d-4313-a5b3-b1b3940af330 None admin] PATH_INFO: `/v3/users/09e7fd26e414495691cc851df63896a7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:48.230549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9dd3a4-344d-4313-a5b3-b1b3940af330 None admin] RBAC: Authorizing `identity:get_user(user_id=09e7fd26e414495691cc851df63896a7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:48.232280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9dd3a4-344d-4313-a5b3-b1b3940af330 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:48.235213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 117/231] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:47 2022] GET /identity/v3/users/09e7fd26e414495691cc851df63896a7 => generated 283 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:48.242593 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dd3a4-344d-4313-a5b3-b1b3940af330 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:48.272461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 115/232] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:29:48 2022] PUT /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d/users/09e7fd26e414495691cc851df63896a7/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:29:48.317398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:48.370525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9dd3a4-344d-4313-a5b3-b1b3940af330 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:48.411075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28bbf7-9703-434e-81c8-e33cc860387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:48.412280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28bbf7-9703-434e-81c8-e33cc860387c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:48.412766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28bbf7-9703-434e-81c8-e33cc860387c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:48.413286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28bbf7-9703-434e-81c8-e33cc860387c None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:48.414719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28bbf7-9703-434e-81c8-e33cc860387c None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:48.416472 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28bbf7-9703-434e-81c8-e33cc860387c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:48.418977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 118/233] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:48 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:48.425675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76985ce2-557f-4410-901a-e5b397c0d00a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:48.570117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76985ce2-557f-4410-901a-e5b397c0d00a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:48.609519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee4609-3833-4eca-9a72-f5e72743539c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:48.610425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee4609-3833-4eca-9a72-f5e72743539c None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:48.610710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee4609-3833-4eca-9a72-f5e72743539c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:48.610989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee4609-3833-4eca-9a72-f5e72743539c None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d/users/09e7fd26e414495691cc851df63896a7/roles/c465d586f0ec4b59815b23087d757fd2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:48.613294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee4609-3833-4eca-9a72-f5e72743539c None admin] RBAC: Authorizing `identity:create_grant(project_id=c5de3bc9e442415487283bf9414ae97d, user_id=09e7fd26e414495691cc851df63896a7, role_id=c465d586f0ec4b59815b23087d757fd2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:48.614769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee4609-3833-4eca-9a72-f5e72743539c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:48.617036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:48.729643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 116/234] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:29:48 2022] PUT /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d/users/09e7fd26e414495691cc851df63896a7/roles/c465d586f0ec4b59815b23087d757fd2 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:29:49.198061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5320-b7f5-40e5-b857-af83cf2ddad8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:49.198561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5320-b7f5-40e5-b857-af83cf2ddad8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:49.198665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5320-b7f5-40e5-b857-af83cf2ddad8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:49.199993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 119/235] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:49.205507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527df29-5458-4f71-80a1-fa125f5ece98 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:49.205932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527df29-5458-4f71-80a1-fa125f5ece98 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:49.206109 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527df29-5458-4f71-80a1-fa125f5ece98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:49.224249 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1527df29-5458-4f71-80a1-fa125f5ece98 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:49.225466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1527df29-5458-4f71-80a1-fa125f5ece98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:49.315194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 117/236] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:49.337819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d5320-b7f5-40e5-b857-af83cf2ddad8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:49.365441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f6890-b2d7-4424-9b0a-f73ad8de9ccd None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:49.365741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f6890-b2d7-4424-9b0a-f73ad8de9ccd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:49.366029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f6890-b2d7-4424-9b0a-f73ad8de9ccd None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:49.367451 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 118/237] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:29:49.380846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8d178-9250-4b19-98a1-84f31047c4d7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:49.381168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8d178-9250-4b19-98a1-84f31047c4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:49.381455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8d178-9250-4b19-98a1-84f31047c4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:49.412716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-15b8d178-9250-4b19-98a1-84f31047c4d7 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:49.416328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-15b8d178-9250-4b19-98a1-84f31047c4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:49.497401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d5320-b7f5-40e5-b857-af83cf2ddad8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:49.540868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:49.557039 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 119/238] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:49.575412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee00e3c-31f5-4d86-abaa-4550de45f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:49.576099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8d178-9250-4b19-98a1-84f31047c4d7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:49.576252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00e3c-31f5-4d86-abaa-4550de45f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:49.576550 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00e3c-31f5-4d86-abaa-4550de45f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:49.576844 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00e3c-31f5-4d86-abaa-4550de45f9e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:49.598411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee00e3c-31f5-4d86-abaa-4550de45f9e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-cee00e3c-31f5-4d86-abaa-4550de45f9e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:49.613062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:49.613974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:49.614772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:49.615355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 120/239] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:49.623362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee00e3c-31f5-4d86-abaa-4550de45f9e9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:49.677245 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:49.771599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8d178-9250-4b19-98a1-84f31047c4d7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:49.851868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445e6af-01a8-46aa-9f84-94ae6e4c78fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:49.852700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445e6af-01a8-46aa-9f84-94ae6e4c78fb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:49.853018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445e6af-01a8-46aa-9f84-94ae6e4c78fb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:49.853351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445e6af-01a8-46aa-9f84-94ae6e4c78fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:49.858717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee00e3c-31f5-4d86-abaa-4550de45f9e9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:49.861835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445e6af-01a8-46aa-9f84-94ae6e4c78fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:49.863380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445e6af-01a8-46aa-9f84-94ae6e4c78fb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:49.865188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 120/240] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:49.887293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445e6af-01a8-46aa-9f84-94ae6e4c78fb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:49.918871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7629fe-c828-448e-a874-1a09d83137f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:49.919694 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7629fe-c828-448e-a874-1a09d83137f7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:49.920023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7629fe-c828-448e-a874-1a09d83137f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:49.921043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7629fe-c828-448e-a874-1a09d83137f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:49.921813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7629fe-c828-448e-a874-1a09d83137f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:49.922119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7629fe-c828-448e-a874-1a09d83137f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:49.923725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7629fe-c828-448e-a874-1a09d83137f7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:49.937932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 121/241] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:49.939380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e80f2-f843-4a4d-8e29-5d3501614d08 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:49.939686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e80f2-f843-4a4d-8e29-5d3501614d08 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:49.939966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e80f2-f843-4a4d-8e29-5d3501614d08 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:49.948901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 122/242] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:49.950030 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e80f2-f843-4a4d-8e29-5d3501614d08 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:50.008535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445e6af-01a8-46aa-9f84-94ae6e4c78fb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:50.013505 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:50.034917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:50.047464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a77d8-d812-4e1f-90ba-02b14da24fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:50.048336 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a77d8-d812-4e1f-90ba-02b14da24fdd None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:50.048772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a77d8-d812-4e1f-90ba-02b14da24fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:50.049242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a77d8-d812-4e1f-90ba-02b14da24fdd None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:50.050116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a77d8-d812-4e1f-90ba-02b14da24fdd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:50.051772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a77d8-d812-4e1f-90ba-02b14da24fdd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:50.081823 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e80f2-f843-4a4d-8e29-5d3501614d08 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:50.118314 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91bdf12-1928-4c9f-b36a-23185b40e99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:50.119451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bdf12-1928-4c9f-b36a-23185b40e99b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:50.119736 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bdf12-1928-4c9f-b36a-23185b40e99b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:50.120010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91bdf12-1928-4c9f-b36a-23185b40e99b None admin] PATH_INFO: `/v3/users/0f37a10fe8ea48c68c9e55516732e26c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:50.134726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91bdf12-1928-4c9f-b36a-23185b40e99b None admin] RBAC: Authorizing `identity:get_user(user_id=0f37a10fe8ea48c68c9e55516732e26c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:50.136192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91bdf12-1928-4c9f-b36a-23185b40e99b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:50.138252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 123/243] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:49 2022] GET /identity/v3/users/0f37a10fe8ea48c68c9e55516732e26c => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:50.139312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a96202-7c8c-4413-a140-0883006d6d75 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:50.139719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a96202-7c8c-4413-a140-0883006d6d75 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:50.140004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a96202-7c8c-4413-a140-0883006d6d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:50.150576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 121/244] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:29:49 2022] POST /identity/v3/users => generated 293 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:29:50.156958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a77d8-d812-4e1f-90ba-02b14da24fdd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:50.165547 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a8a96202-7c8c-4413-a140-0883006d6d75 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:50.167577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a8a96202-7c8c-4413-a140-0883006d6d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:50.320210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 124/245] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:50.340846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a77d8-d812-4e1f-90ba-02b14da24fdd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:50.373172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a96202-7c8c-4413-a140-0883006d6d75 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:50.406485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccae25d-5d68-4c00-be3b-5e0ebb6e2e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:50.407150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccae25d-5d68-4c00-be3b-5e0ebb6e2e96 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:50.407434 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccae25d-5d68-4c00-be3b-5e0ebb6e2e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:50.407709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccae25d-5d68-4c00-be3b-5e0ebb6e2e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:50.412574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:50.461863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:50.485639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccae25d-5d68-4c00-be3b-5e0ebb6e2e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:50.487150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccae25d-5d68-4c00-be3b-5e0ebb6e2e96 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:50.491083 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a96202-7c8c-4413-a140-0883006d6d75 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:50.530380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03593444-5d43-4093-887e-e6e282cf18d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:50.531573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03593444-5d43-4093-887e-e6e282cf18d5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:50.532130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03593444-5d43-4093-887e-e6e282cf18d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:50.532600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03593444-5d43-4093-887e-e6e282cf18d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:50.545320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03593444-5d43-4093-887e-e6e282cf18d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-03593444-5d43-4093-887e-e6e282cf18d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:50.553699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:50.554554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:50.555328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:50.557747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 125/246] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:50.563671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03593444-5d43-4093-887e-e6e282cf18d5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:50.603278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 122/247] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:50.609567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccae25d-5d68-4c00-be3b-5e0ebb6e2e96 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:50.673138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:50.682647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03593444-5d43-4093-887e-e6e282cf18d5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:50.694209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:50.717918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccae25d-5d68-4c00-be3b-5e0ebb6e2e96 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:50.718824 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4581d7-765b-4ecf-9a9a-de0c5bc29e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:50.719626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4581d7-765b-4ecf-9a9a-de0c5bc29e59 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:50.720045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4581d7-765b-4ecf-9a9a-de0c5bc29e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:50.720386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4581d7-765b-4ecf-9a9a-de0c5bc29e59 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:50.721832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4581d7-765b-4ecf-9a9a-de0c5bc29e59 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:50.721832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4581d7-765b-4ecf-9a9a-de0c5bc29e59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:50.724390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4581d7-765b-4ecf-9a9a-de0c5bc29e59 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:50.738922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 126/248] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:50.741118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4581d7-765b-4ecf-9a9a-de0c5bc29e59 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:50.767142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761a8e9-e994-447b-8277-7de3105043ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:50.768025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761a8e9-e994-447b-8277-7de3105043ee None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:50.768367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761a8e9-e994-447b-8277-7de3105043ee None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:50.768651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761a8e9-e994-447b-8277-7de3105043ee None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:50.769950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761a8e9-e994-447b-8277-7de3105043ee None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:50.774229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761a8e9-e994-447b-8277-7de3105043ee None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:50.776311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 123/249] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:50 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:50.782471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761a8e9-e994-447b-8277-7de3105043ee None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:50.860265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4581d7-765b-4ecf-9a9a-de0c5bc29e59 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:50.906000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3f240-58bd-46e7-9f4d-446e952c97ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:50.906895 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3f240-58bd-46e7-9f4d-446e952c97ff None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:50.907175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3f240-58bd-46e7-9f4d-446e952c97ff None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:50.907449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3f240-58bd-46e7-9f4d-446e952c97ff None admin] PATH_INFO: `/v3/users/2c209b7964d54419b4f8395278e9aff9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:50.908772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3f240-58bd-46e7-9f4d-446e952c97ff None admin] RBAC: Authorizing `identity:get_user(user_id=2c209b7964d54419b4f8395278e9aff9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:50.910292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3f240-58bd-46e7-9f4d-446e952c97ff None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:50.912935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 127/250] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:50 2022] GET /identity/v3/users/2c209b7964d54419b4f8395278e9aff9 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:50.914375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751995-7b92-4a98-aa2b-9ba0e46658ab None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:50.914375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751995-7b92-4a98-aa2b-9ba0e46658ab None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:50.914375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751995-7b92-4a98-aa2b-9ba0e46658ab None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:50.915306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 128/251] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:50.917854 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2751995-7b92-4a98-aa2b-9ba0e46658ab None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:50.920063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761a8e9-e994-447b-8277-7de3105043ee None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:50.965788 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:50.973737 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c55a81-bbc0-4cec-9dd0-ed5aba7039c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:50.974445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55a81-bbc0-4cec-9dd0-ed5aba7039c4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:50.974695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55a81-bbc0-4cec-9dd0-ed5aba7039c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:50.974976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c55a81-bbc0-4cec-9dd0-ed5aba7039c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:50.975835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c55a81-bbc0-4cec-9dd0-ed5aba7039c4 None admin] RBAC: Adding query filter params (role.id=4f99bdbe513b4500b96f9c530dac52ed, scope.project.id=c5de3bc9e442415487283bf9414ae97d, user.id=0f37a10fe8ea48c68c9e55516732e26c) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:50.983774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c55a81-bbc0-4cec-9dd0-ed5aba7039c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:50.983774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c55a81-bbc0-4cec-9dd0-ed5aba7039c4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:50.986410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 124/252] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:50 2022] GET /identity/v3/role_assignments?user.id=0f37a10fe8ea48c68c9e55516732e26c&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=4f99bdbe513b4500b96f9c530dac52ed => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:50.988057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aade01-5fc2-45f2-8abf-c6428669e7d3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:50.988351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aade01-5fc2-45f2-8abf-c6428669e7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:50.988871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aade01-5fc2-45f2-8abf-c6428669e7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:51.009631 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b4aade01-5fc2-45f2-8abf-c6428669e7d3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:51.010860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b4aade01-5fc2-45f2-8abf-c6428669e7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:51.016252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:51.065623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2751995-7b92-4a98-aa2b-9ba0e46658ab None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:51.084799 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 125/253] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:51.108034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a679e0-6596-4133-af02-927c2d7fb5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:51.108959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a679e0-6596-4133-af02-927c2d7fb5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:51.109273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aade01-5fc2-45f2-8abf-c6428669e7d3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:51.109382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a679e0-6596-4133-af02-927c2d7fb5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:51.109654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a679e0-6596-4133-af02-927c2d7fb5a9 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:51.110951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a679e0-6596-4133-af02-927c2d7fb5a9 None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:51.117157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a679e0-6596-4133-af02-927c2d7fb5a9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:51.120129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 129/254] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:50 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:51.125919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a679e0-6596-4133-af02-927c2d7fb5a9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:51.304866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aade01-5fc2-45f2-8abf-c6428669e7d3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:51.319863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:51.342561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a679e0-6596-4133-af02-927c2d7fb5a9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:51.377661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429983c5-a1c7-4ed8-9d75-6a922e5c4c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:51.379112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429983c5-a1c7-4ed8-9d75-6a922e5c4c7e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:51.379857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429983c5-a1c7-4ed8-9d75-6a922e5c4c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:51.381173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429983c5-a1c7-4ed8-9d75-6a922e5c4c7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:51.400004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:51.404029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429983c5-a1c7-4ed8-9d75-6a922e5c4c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-429983c5-a1c7-4ed8-9d75-6a922e5c4c7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:51.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:51.419368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 126/255] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:29:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:51.420127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe79768-7e92-4e03-b38f-1d37651774f6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:51.421572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe79768-7e92-4e03-b38f-1d37651774f6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:51.421572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe79768-7e92-4e03-b38f-1d37651774f6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:51.421572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 127/256] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:51.424082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572fb43-0274-478c-9cf6-4185a21f6df1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:51.424278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572fb43-0274-478c-9cf6-4185a21f6df1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:51.424567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572fb43-0274-478c-9cf6-4185a21f6df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:51.426128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35e505-986e-4d65-bbfa-3ee9af33f1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:51.427078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35e505-986e-4d65-bbfa-3ee9af33f1b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:51.427362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35e505-986e-4d65-bbfa-3ee9af33f1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:51.427634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35e505-986e-4d65-bbfa-3ee9af33f1b2 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d/users/2c209b7964d54419b4f8395278e9aff9/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:51.434367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35e505-986e-4d65-bbfa-3ee9af33f1b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5de3bc9e442415487283bf9414ae97d, user_id=2c209b7964d54419b4f8395278e9aff9, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:51.436043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35e505-986e-4d65-bbfa-3ee9af33f1b2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:51.442349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3572fb43-0274-478c-9cf6-4185a21f6df1 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:51.443555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3572fb43-0274-478c-9cf6-4185a21f6df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:51.518180 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 128/257] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:51.518864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572fb43-0274-478c-9cf6-4185a21f6df1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:51.535463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 130/258] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:29:51 2022] PUT /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d/users/2c209b7964d54419b4f8395278e9aff9/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:29:51.589845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35e505-986e-4d65-bbfa-3ee9af33f1b2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:51.677701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572fb43-0274-478c-9cf6-4185a21f6df1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:51.711816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:51.719227 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80892089-cb6c-4a79-835a-e65e3a6020a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:51.719926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80892089-cb6c-4a79-835a-e65e3a6020a6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:51.720212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80892089-cb6c-4a79-835a-e65e3a6020a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:51.720493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80892089-cb6c-4a79-835a-e65e3a6020a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:51.721341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80892089-cb6c-4a79-835a-e65e3a6020a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:51.721653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80892089-cb6c-4a79-835a-e65e3a6020a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:51.724024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80892089-cb6c-4a79-835a-e65e3a6020a6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:51.740482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 129/259] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:51.747179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80892089-cb6c-4a79-835a-e65e3a6020a6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:51.801606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35e505-986e-4d65-bbfa-3ee9af33f1b2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:51.883544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f6c08-9d63-46da-9d06-237517aebfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:51.884385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f6c08-9d63-46da-9d06-237517aebfa2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:51.884672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f6c08-9d63-46da-9d06-237517aebfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:51.885186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f6c08-9d63-46da-9d06-237517aebfa2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:51.897774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f6c08-9d63-46da-9d06-237517aebfa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-451f6c08-9d63-46da-9d06-237517aebfa2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:51.906964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:51.907937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:51.908723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 131/260] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:29:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:51.925451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f6c08-9d63-46da-9d06-237517aebfa2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:51.945035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:51.997120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80892089-cb6c-4a79-835a-e65e3a6020a6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:52.055936 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f6c08-9d63-46da-9d06-237517aebfa2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:52.058776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba034d54-3294-461b-beb8-657b99a05b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:52.059674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba034d54-3294-461b-beb8-657b99a05b3a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:52.059958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba034d54-3294-461b-beb8-657b99a05b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:52.060241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba034d54-3294-461b-beb8-657b99a05b3a None admin] PATH_INFO: `/v3/users/09e7fd26e414495691cc851df63896a7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:52.063985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba034d54-3294-461b-beb8-657b99a05b3a None admin] RBAC: Authorizing `identity:get_user(user_id=09e7fd26e414495691cc851df63896a7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:52.065608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba034d54-3294-461b-beb8-657b99a05b3a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:52.067417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 130/261] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:51 2022] GET /identity/v3/users/09e7fd26e414495691cc851df63896a7 => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:52.072845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba034d54-3294-461b-beb8-657b99a05b3a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:52.093144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb199a-12ef-4feb-adb0-7200267b12dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:52.094384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb199a-12ef-4feb-adb0-7200267b12dc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:52.094870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb199a-12ef-4feb-adb0-7200267b12dc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:52.095224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb199a-12ef-4feb-adb0-7200267b12dc None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:52.095998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb199a-12ef-4feb-adb0-7200267b12dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:52.096307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb199a-12ef-4feb-adb0-7200267b12dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:52.097744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb199a-12ef-4feb-adb0-7200267b12dc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:52.106419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 132/262] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:29:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:52.112107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb199a-12ef-4feb-adb0-7200267b12dc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:52.187007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:52.197230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:52.220196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb199a-12ef-4feb-adb0-7200267b12dc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:52.263997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b42c2-8cc4-45a8-9463-d129c810b7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:52.265517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b42c2-8cc4-45a8-9463-d129c810b7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:52.265870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b42c2-8cc4-45a8-9463-d129c810b7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:52.266211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b42c2-8cc4-45a8-9463-d129c810b7c8 None admin] PATH_INFO: `/v3/users/09e7fd26e414495691cc851df63896a7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:52.267545 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b42c2-8cc4-45a8-9463-d129c810b7c8 None admin] RBAC: Authorizing `identity:get_user(user_id=09e7fd26e414495691cc851df63896a7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:52.269117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b42c2-8cc4-45a8-9463-d129c810b7c8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:52.273592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 133/263] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:52 2022] GET /identity/v3/users/09e7fd26e414495691cc851df63896a7 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:52.278091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b42c2-8cc4-45a8-9463-d129c810b7c8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:52.298349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba034d54-3294-461b-beb8-657b99a05b3a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:52.369586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac663d8f-3674-4802-9649-7e2dfec310fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:52.370262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac663d8f-3674-4802-9649-7e2dfec310fd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:52.370662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac663d8f-3674-4802-9649-7e2dfec310fd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:52.370943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac663d8f-3674-4802-9649-7e2dfec310fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:52.396479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b42c2-8cc4-45a8-9463-d129c810b7c8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:52.433264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45f34e-8c37-4f6d-b0f1-b5c239a4b6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:52.433925 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45f34e-8c37-4f6d-b0f1-b5c239a4b6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:52.434212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45f34e-8c37-4f6d-b0f1-b5c239a4b6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:52.434493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45f34e-8c37-4f6d-b0f1-b5c239a4b6d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:52.445736 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:52.505713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45f34e-8c37-4f6d-b0f1-b5c239a4b6d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:52.507364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45f34e-8c37-4f6d-b0f1-b5c239a4b6d7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:52.517665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac663d8f-3674-4802-9649-7e2dfec310fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:52.519261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac663d8f-3674-4802-9649-7e2dfec310fd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:52.616398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 134/264] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:52.622327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45f34e-8c37-4f6d-b0f1-b5c239a4b6d7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:52.628205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:52.668935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:52.699163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 131/265] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:52.705011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac663d8f-3674-4802-9649-7e2dfec310fd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:52.819868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45f34e-8c37-4f6d-b0f1-b5c239a4b6d7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:52.837034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac663d8f-3674-4802-9649-7e2dfec310fd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:52.908476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ff6ba-2a1a-4761-944b-24ce58553d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:52.909407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ff6ba-2a1a-4761-944b-24ce58553d1c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:52.909691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ff6ba-2a1a-4761-944b-24ce58553d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:52.909973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ff6ba-2a1a-4761-944b-24ce58553d1c None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:52.911189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ff6ba-2a1a-4761-944b-24ce58553d1c None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:52.912653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ff6ba-2a1a-4761-944b-24ce58553d1c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:52.914604 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 132/266] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:52 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:52.920653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ff6ba-2a1a-4761-944b-24ce58553d1c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:52.928935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7472c-896b-47ad-b18a-26b1d28ec90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:52.929833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7472c-896b-47ad-b18a-26b1d28ec90d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:52.930117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7472c-896b-47ad-b18a-26b1d28ec90d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:52.930385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7472c-896b-47ad-b18a-26b1d28ec90d None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:52.942985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7472c-896b-47ad-b18a-26b1d28ec90d None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:52.942985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7472c-896b-47ad-b18a-26b1d28ec90d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:52.942985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 135/267] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:52 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:52.948419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7472c-896b-47ad-b18a-26b1d28ec90d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:52.983359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:53.095809 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ff6ba-2a1a-4761-944b-24ce58553d1c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:53.136278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7472c-896b-47ad-b18a-26b1d28ec90d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:53.153156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:53.191303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc924e2a-199a-486f-b0df-107b2e7b07d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:53.192031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc924e2a-199a-486f-b0df-107b2e7b07d1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:53.192326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc924e2a-199a-486f-b0df-107b2e7b07d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:53.192605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc924e2a-199a-486f-b0df-107b2e7b07d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:53.193487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc924e2a-199a-486f-b0df-107b2e7b07d1 None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=c5de3bc9e442415487283bf9414ae97d, user.id=09e7fd26e414495691cc851df63896a7) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:53.193801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc924e2a-199a-486f-b0df-107b2e7b07d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:53.195277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc924e2a-199a-486f-b0df-107b2e7b07d1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:53.195813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4450b-988c-4252-bccc-5548299af53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:53.196487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4450b-988c-4252-bccc-5548299af53c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:53.196771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4450b-988c-4252-bccc-5548299af53c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:53.197087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4450b-988c-4252-bccc-5548299af53c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:53.197940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4450b-988c-4252-bccc-5548299af53c None admin] RBAC: Adding query filter params (role.id=c465d586f0ec4b59815b23087d757fd2, scope.project.id=c5de3bc9e442415487283bf9414ae97d, user.id=09e7fd26e414495691cc851df63896a7) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:53.198251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4450b-988c-4252-bccc-5548299af53c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:53.199700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b4450b-988c-4252-bccc-5548299af53c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:53.213025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 136/268] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:52 2022] GET /identity/v3/role_assignments?user.id=09e7fd26e414495691cc851df63896a7&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=c465d586f0ec4b59815b23087d757fd2 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:53.223434 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 133/269] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:52 2022] GET /identity/v3/role_assignments?user.id=09e7fd26e414495691cc851df63896a7&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=81457691f3a2456fa33358cb6a922d05 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:53.726047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c5696-1d9f-4733-b9b0-2a00f6ec17a9 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:53.726555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c5696-1d9f-4733-b9b0-2a00f6ec17a9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:53.726630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c5696-1d9f-4733-b9b0-2a00f6ec17a9 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:53.728311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 137/270] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:53.735001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57d91e-c363-404c-8957-e407125088d6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:53.735502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57d91e-c363-404c-8957-e407125088d6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:53.735578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57d91e-c363-404c-8957-e407125088d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:53.759225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1a57d91e-c363-404c-8957-e407125088d6 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:53.760436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1a57d91e-c363-404c-8957-e407125088d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:53.843372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 134/271] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:53.868138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3150-8ce9-483f-8de5-e18351b24b0f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:53.868696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3150-8ce9-483f-8de5-e18351b24b0f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:53.869163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3150-8ce9-483f-8de5-e18351b24b0f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:53.871024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 138/272] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:53.873843 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57d91e-c363-404c-8957-e407125088d6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:53.876379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5709415-efd2-42d1-934b-5ad48fcb8dea None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:53.876916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5709415-efd2-42d1-934b-5ad48fcb8dea None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:53.877404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5709415-efd2-42d1-934b-5ad48fcb8dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:53.895808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b5709415-efd2-42d1-934b-5ad48fcb8dea None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:53.897414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b5709415-efd2-42d1-934b-5ad48fcb8dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:53.902344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:53.977979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 139/273] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:53.987768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f24ca-262c-42b3-bd2f-00606c980264 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:53.988266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f24ca-262c-42b3-bd2f-00606c980264 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:53.988692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f24ca-262c-42b3-bd2f-00606c980264 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:53.990848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 140/274] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:29:53.997270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0dbf5-3998-4243-9ac0-6d570297f312 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:53.997728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0dbf5-3998-4243-9ac0-6d570297f312 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:53.998149 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0dbf5-3998-4243-9ac0-6d570297f312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.009940 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57d91e-c363-404c-8957-e407125088d6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:54.016302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-20e0dbf5-3998-4243-9ac0-6d570297f312 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:54.017544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-20e0dbf5-3998-4243-9ac0-6d570297f312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:54.052295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c4643-bb95-4759-9118-198cb3467b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:54.053150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c4643-bb95-4759-9118-198cb3467b2c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:54.053440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c4643-bb95-4759-9118-198cb3467b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:54.053720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c4643-bb95-4759-9118-198cb3467b2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.054980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c4643-bb95-4759-9118-198cb3467b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:54.056462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c4643-bb95-4759-9118-198cb3467b2c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:54.064333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 135/275] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:54.064333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c4643-bb95-4759-9118-198cb3467b2c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:54.100948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 141/276] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:54.113118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0dbf5-3998-4243-9ac0-6d570297f312 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:54.144307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:54.166447 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:54.177759 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c4643-bb95-4759-9118-198cb3467b2c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:54.214655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f87581-887a-4b39-a002-f3fbd06ecfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:54.215451 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f87581-887a-4b39-a002-f3fbd06ecfae None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:54.215735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f87581-887a-4b39-a002-f3fbd06ecfae None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:54.216005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f87581-887a-4b39-a002-f3fbd06ecfae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.216360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0dbf5-3998-4243-9ac0-6d570297f312 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:54.226816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f87581-887a-4b39-a002-f3fbd06ecfae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-e6f87581-887a-4b39-a002-f3fbd06ecfae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:54.236032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:54.237268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:54.238220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:54.246529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 136/277] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:54.247509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f87581-887a-4b39-a002-f3fbd06ecfae None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:54.260563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca37ea-b103-4159-a136-cde7a9a3f0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:54.261570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca37ea-b103-4159-a136-cde7a9a3f0a4 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:54.261843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca37ea-b103-4159-a136-cde7a9a3f0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:54.262120 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca37ea-b103-4159-a136-cde7a9a3f0a4 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.262833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca37ea-b103-4159-a136-cde7a9a3f0a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:54.265050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca37ea-b103-4159-a136-cde7a9a3f0a4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:54.354141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f87581-887a-4b39-a002-f3fbd06ecfae None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:54.389660 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c92755-65a0-4c43-ab71-ba15c0331e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:54.390528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c92755-65a0-4c43-ab71-ba15c0331e7f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:54.390912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c92755-65a0-4c43-ab71-ba15c0331e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:54.391014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c92755-65a0-4c43-ab71-ba15c0331e7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.398253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 142/278] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:29:54 2022] POST /identity/v3/users => generated 293 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:29:54.402604 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c92755-65a0-4c43-ab71-ba15c0331e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:54.409957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca37ea-b103-4159-a136-cde7a9a3f0a4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-92c92755-65a0-4c43-ab71-ba15c0331e7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:54.410538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:54.418293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 137/279] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:54.419170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c92755-65a0-4c43-ab71-ba15c0331e7f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:54.437817 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:54.547782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c92755-65a0-4c43-ab71-ba15c0331e7f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:54.549107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:54.565301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca37ea-b103-4159-a136-cde7a9a3f0a4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:54.586633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd176b-a24c-4052-9a17-09f3b6cc2cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:54.587565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd176b-a24c-4052-9a17-09f3b6cc2cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:54.588680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd176b-a24c-4052-9a17-09f3b6cc2cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:54.588680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd176b-a24c-4052-9a17-09f3b6cc2cb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.594596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd176b-a24c-4052-9a17-09f3b6cc2cb3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:54.594596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd176b-a24c-4052-9a17-09f3b6cc2cb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:54.594596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd176b-a24c-4052-9a17-09f3b6cc2cb3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:54.603815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea85da0-e7bb-4aa6-b491-9e81a099e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:54.604547 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea85da0-e7bb-4aa6-b491-9e81a099e399 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:54.604840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea85da0-e7bb-4aa6-b491-9e81a099e399 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:54.605148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea85da0-e7bb-4aa6-b491-9e81a099e399 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.605688 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 138/280] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:54.607296 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fba3d-4497-4559-b9f3-cb5e52eab825 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:54.607296 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fba3d-4497-4559-b9f3-cb5e52eab825 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:54.607562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fba3d-4497-4559-b9f3-cb5e52eab825 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.609741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 139/281] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:54.611312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea85da0-e7bb-4aa6-b491-9e81a099e399 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:54.611626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea85da0-e7bb-4aa6-b491-9e81a099e399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:54.611922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fba3d-4497-4559-b9f3-cb5e52eab825 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:54.613146 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea85da0-e7bb-4aa6-b491-9e81a099e399 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:54.625100 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 143/282] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:54.628475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67c9ce-e42c-4261-8029-eb4f651b1e8e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:54.629491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67c9ce-e42c-4261-8029-eb4f651b1e8e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:54.629951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67c9ce-e42c-4261-8029-eb4f651b1e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.656363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9e67c9ce-e42c-4261-8029-eb4f651b1e8e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:54.660038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9e67c9ce-e42c-4261-8029-eb4f651b1e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:54.715148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:54.740446 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 144/283] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:54.741376 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67c9ce-e42c-4261-8029-eb4f651b1e8e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:54.751415 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fba3d-4497-4559-b9f3-cb5e52eab825 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:54.794740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444cac6-8d01-4639-b601-05c87c459adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:54.795723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444cac6-8d01-4639-b601-05c87c459adf None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:54.795997 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444cac6-8d01-4639-b601-05c87c459adf None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:54.796278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444cac6-8d01-4639-b601-05c87c459adf None admin] PATH_INFO: `/v3/users/0f37a10fe8ea48c68c9e55516732e26c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.797667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444cac6-8d01-4639-b601-05c87c459adf None admin] RBAC: Authorizing `identity:get_user(user_id=0f37a10fe8ea48c68c9e55516732e26c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:54.799201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444cac6-8d01-4639-b601-05c87c459adf None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:54.801458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 140/284] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:54 2022] GET /identity/v3/users/0f37a10fe8ea48c68c9e55516732e26c => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:54.802204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444cac6-8d01-4639-b601-05c87c459adf None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:54.843957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:54.864841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67c9ce-e42c-4261-8029-eb4f651b1e8e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:54.903756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc86ded-f76b-42ae-9eae-4530dbac9be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:54.905962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc86ded-f76b-42ae-9eae-4530dbac9be6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:54.906522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc86ded-f76b-42ae-9eae-4530dbac9be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:54.906999 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc86ded-f76b-42ae-9eae-4530dbac9be6 None admin] PATH_INFO: `/v3/users/c4ce363db1b6428e89ebf4125deb87ab` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.918158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444cac6-8d01-4639-b601-05c87c459adf None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:54.929407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc86ded-f76b-42ae-9eae-4530dbac9be6 None admin] RBAC: Authorizing `identity:get_user(user_id=c4ce363db1b6428e89ebf4125deb87ab)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:54.931987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc86ded-f76b-42ae-9eae-4530dbac9be6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:54.935038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 145/285] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:54 2022] GET /identity/v3/users/c4ce363db1b6428e89ebf4125deb87ab => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:54.936210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc86ded-f76b-42ae-9eae-4530dbac9be6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:54.967479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a884bc-9342-495b-baa4-92e55aa1245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:54.968955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a884bc-9342-495b-baa4-92e55aa1245a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:54.969318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a884bc-9342-495b-baa4-92e55aa1245a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:54.969817 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a884bc-9342-495b-baa4-92e55aa1245a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:54.980196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:54.986089 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a884bc-9342-495b-baa4-92e55aa1245a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-86a884bc-9342-495b-baa4-92e55aa1245a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:54.995327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:54.996449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:54.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 141/286] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:55.003652 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a884bc-9342-495b-baa4-92e55aa1245a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:55.071047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc86ded-f76b-42ae-9eae-4530dbac9be6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:55.117565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d08e81-dc52-4e20-b49d-b5ee7f987df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:55.119545 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d08e81-dc52-4e20-b49d-b5ee7f987df0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:55.119976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d08e81-dc52-4e20-b49d-b5ee7f987df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:55.120393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d08e81-dc52-4e20-b49d-b5ee7f987df0 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:55.121467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a884bc-9342-495b-baa4-92e55aa1245a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:55.121979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d08e81-dc52-4e20-b49d-b5ee7f987df0 None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:55.124352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d08e81-dc52-4e20-b49d-b5ee7f987df0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:55.127032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 146/287] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:54 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:55.127492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d08e81-dc52-4e20-b49d-b5ee7f987df0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:55.138241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:55.163527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195f75-f4b8-412a-b300-e17e4c6490a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:55.164356 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195f75-f4b8-412a-b300-e17e4c6490a1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:55.164752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195f75-f4b8-412a-b300-e17e4c6490a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:55.165073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24195f75-f4b8-412a-b300-e17e4c6490a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:55.165916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24195f75-f4b8-412a-b300-e17e4c6490a1 None admin] RBAC: Adding query filter params (role.id=3b2e1fad42f441e9a61c142477a50b11, user.id=c4ce363db1b6428e89ebf4125deb87ab, scope.system=all) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:55.166233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24195f75-f4b8-412a-b300-e17e4c6490a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:55.167878 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24195f75-f4b8-412a-b300-e17e4c6490a1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:55.175328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 142/288] 10.222.0.88 () {56 vars in 1308 bytes} [Mon Jul 18 16:29:54 2022] GET /identity/v3/role_assignments?user.id=c4ce363db1b6428e89ebf4125deb87ab&scope.system=all&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:55.176705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195f75-f4b8-412a-b300-e17e4c6490a1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:55.249509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d08e81-dc52-4e20-b49d-b5ee7f987df0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:55.270315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:55.290606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3b208-49de-4310-b563-1003f266ebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:55.291756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3b208-49de-4310-b563-1003f266ebea None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:55.292276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3b208-49de-4310-b563-1003f266ebea None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:55.292833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3b208-49de-4310-b563-1003f266ebea None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:55.293872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3b208-49de-4310-b563-1003f266ebea None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:55.294404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3b208-49de-4310-b563-1003f266ebea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:55.296761 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3b208-49de-4310-b563-1003f266ebea None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:55.307000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 147/289] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:55.314094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3b208-49de-4310-b563-1003f266ebea None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:55.323805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24195f75-f4b8-412a-b300-e17e4c6490a1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:55.371642 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a4324-dca7-4e97-8ede-8817658223a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:55.372565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4324-dca7-4e97-8ede-8817658223a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:55.372850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4324-dca7-4e97-8ede-8817658223a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:55.373177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a4324-dca7-4e97-8ede-8817658223a4 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d/users/0f37a10fe8ea48c68c9e55516732e26c/roles/4f99bdbe513b4500b96f9c530dac52ed` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:55.377129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a4324-dca7-4e97-8ede-8817658223a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5de3bc9e442415487283bf9414ae97d, user_id=0f37a10fe8ea48c68c9e55516732e26c, role_id=4f99bdbe513b4500b96f9c530dac52ed)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:55.379034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a4324-dca7-4e97-8ede-8817658223a4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:55.411696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:55.455707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3b208-49de-4310-b563-1003f266ebea None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:55.464284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 143/290] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:29:55 2022] PUT /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d/users/0f37a10fe8ea48c68c9e55516732e26c/roles/4f99bdbe513b4500b96f9c530dac52ed => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:29:55.499014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32676fee-32a7-47be-aff8-a7f28de45912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:55.500357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32676fee-32a7-47be-aff8-a7f28de45912 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:55.500849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32676fee-32a7-47be-aff8-a7f28de45912 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:55.501382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32676fee-32a7-47be-aff8-a7f28de45912 None admin] PATH_INFO: `/v3/users/2c209b7964d54419b4f8395278e9aff9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:55.502874 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32676fee-32a7-47be-aff8-a7f28de45912 None admin] RBAC: Authorizing `identity:get_user(user_id=2c209b7964d54419b4f8395278e9aff9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:55.504643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32676fee-32a7-47be-aff8-a7f28de45912 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:55.507039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 148/291] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:55 2022] GET /identity/v3/users/2c209b7964d54419b4f8395278e9aff9 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:55.512574 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a4324-dca7-4e97-8ede-8817658223a4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:55.627804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a4324-dca7-4e97-8ede-8817658223a4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:55.658744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:55.670701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bbbae-1945-4f35-8e20-bfa89e31625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:55.672735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bbbae-1945-4f35-8e20-bfa89e31625a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:55.672735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bbbae-1945-4f35-8e20-bfa89e31625a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:55.672735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bbbae-1945-4f35-8e20-bfa89e31625a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:55.751462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bbbae-1945-4f35-8e20-bfa89e31625a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:55.753010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bbbae-1945-4f35-8e20-bfa89e31625a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:55.884821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 144/292] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:29:55.885779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32676fee-32a7-47be-aff8-a7f28de45912 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:56.030611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32676fee-32a7-47be-aff8-a7f28de45912 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:56.078973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e787b-b32b-4c2a-971d-4e8644ea82bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:56.079853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e787b-b32b-4c2a-971d-4e8644ea82bb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:56.080140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e787b-b32b-4c2a-971d-4e8644ea82bb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:56.080415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e787b-b32b-4c2a-971d-4e8644ea82bb None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:56.081670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e787b-b32b-4c2a-971d-4e8644ea82bb None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:56.083143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e787b-b32b-4c2a-971d-4e8644ea82bb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:56.084909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 149/293] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:55 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:56.090867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bbbae-1945-4f35-8e20-bfa89e31625a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:56.117129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:56.211638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bbbae-1945-4f35-8e20-bfa89e31625a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:56.246474 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c9571-0258-4f11-bcc1-797bd0c26207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:56.247142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c9571-0258-4f11-bcc1-797bd0c26207 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:56.247428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c9571-0258-4f11-bcc1-797bd0c26207 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:56.247708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c9571-0258-4f11-bcc1-797bd0c26207 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:56.249108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c9571-0258-4f11-bcc1-797bd0c26207 None admin] RBAC: Adding query filter params (role.id=3b2e1fad42f441e9a61c142477a50b11, scope.project.id=c5de3bc9e442415487283bf9414ae97d, user.id=2c209b7964d54419b4f8395278e9aff9) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:56.249491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c9571-0258-4f11-bcc1-797bd0c26207 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:56.250970 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c9571-0258-4f11-bcc1-797bd0c26207 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:56.261625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 145/294] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:56 2022] GET /identity/v3/role_assignments?user.id=2c209b7964d54419b4f8395278e9aff9&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:56.803787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a3fb-6672-4199-bebb-d8741ec9ad49 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:56.804122 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a3fb-6672-4199-bebb-d8741ec9ad49 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:56.804339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a3fb-6672-4199-bebb-d8741ec9ad49 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:56.806041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 150/295] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:56.812205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80146e36-804b-483e-8df6-84cb74be2fde None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:56.812670 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80146e36-804b-483e-8df6-84cb74be2fde None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:56.812855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80146e36-804b-483e-8df6-84cb74be2fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:56.836865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-80146e36-804b-483e-8df6-84cb74be2fde None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:56.838216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-80146e36-804b-483e-8df6-84cb74be2fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:56.914302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 146/296] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:56.938506 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128a3fb-6672-4199-bebb-d8741ec9ad49 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:56.944559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:57.058053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128a3fb-6672-4199-bebb-d8741ec9ad49 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:57.094223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceb789-8076-4930-a17c-ffd5a8db52c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:57.095033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceb789-8076-4930-a17c-ffd5a8db52c9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.095320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceb789-8076-4930-a17c-ffd5a8db52c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.095599 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceb789-8076-4930-a17c-ffd5a8db52c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.106665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ceb789-8076-4930-a17c-ffd5a8db52c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-92ceb789-8076-4930-a17c-ffd5a8db52c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:57.114518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:57.115400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:57.116186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 151/297] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:57.126018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80146e36-804b-483e-8df6-84cb74be2fde None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:57.133198 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44794094-85e5-4469-b6a0-b6701adbd418 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.133488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44794094-85e5-4469-b6a0-b6701adbd418 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.133773 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44794094-85e5-4469-b6a0-b6701adbd418 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.136375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 152/298] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:29:57.141036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff73b57-c276-4a87-b65d-26976a9a37cd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.141324 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff73b57-c276-4a87-b65d-26976a9a37cd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.141603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff73b57-c276-4a87-b65d-26976a9a37cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.145885 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:57.159654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5ff73b57-c276-4a87-b65d-26976a9a37cd None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:57.161014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5ff73b57-c276-4a87-b65d-26976a9a37cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:57.235745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:57.246691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80146e36-804b-483e-8df6-84cb74be2fde None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:57.252466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 153/299] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:57.275862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff73b57-c276-4a87-b65d-26976a9a37cd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:57.287095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9aca18-73bd-4fb9-8fc6-d1686276e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:57.288009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9aca18-73bd-4fb9-8fc6-d1686276e53f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.288736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9aca18-73bd-4fb9-8fc6-d1686276e53f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.289230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9aca18-73bd-4fb9-8fc6-d1686276e53f None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.290454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9aca18-73bd-4fb9-8fc6-d1686276e53f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:57.290908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9aca18-73bd-4fb9-8fc6-d1686276e53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:57.293135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9aca18-73bd-4fb9-8fc6-d1686276e53f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:57.304601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 147/300] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:57.310692 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9aca18-73bd-4fb9-8fc6-d1686276e53f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:57.406054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff73b57-c276-4a87-b65d-26976a9a37cd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:57.421315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:57.437545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9aca18-73bd-4fb9-8fc6-d1686276e53f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:57.444645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72243e5-a79a-4c13-9376-e4fdb45c9938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:57.445717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72243e5-a79a-4c13-9376-e4fdb45c9938 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.452212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72243e5-a79a-4c13-9376-e4fdb45c9938 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.452212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72243e5-a79a-4c13-9376-e4fdb45c9938 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.452212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72243e5-a79a-4c13-9376-e4fdb45c9938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:57.452212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72243e5-a79a-4c13-9376-e4fdb45c9938 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:57.452212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 154/301] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:57.458479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72243e5-a79a-4c13-9376-e4fdb45c9938 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:57.482612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdec9f-f639-4940-94fa-242bc33830fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:57.484064 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdec9f-f639-4940-94fa-242bc33830fd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.484534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdec9f-f639-4940-94fa-242bc33830fd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.485035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdec9f-f639-4940-94fa-242bc33830fd None admin] PATH_INFO: `/v3/users/f1936b1009f242a592450d07c5bc551a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.506194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdec9f-f639-4940-94fa-242bc33830fd None admin] RBAC: Authorizing `identity:get_user(user_id=f1936b1009f242a592450d07c5bc551a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:57.508074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdec9f-f639-4940-94fa-242bc33830fd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:57.510926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 148/302] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:57 2022] GET /identity/v3/users/f1936b1009f242a592450d07c5bc551a => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:57.517437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdec9f-f639-4940-94fa-242bc33830fd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:57.540016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:57.586648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72243e5-a79a-4c13-9376-e4fdb45c9938 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:57.626055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bb890-c467-4e72-8c79-f1320440a5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:57.626759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bb890-c467-4e72-8c79-f1320440a5bd None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.627317 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bb890-c467-4e72-8c79-f1320440a5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.627426 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bb890-c467-4e72-8c79-f1320440a5bd None admin] PATH_INFO: `/v3/groups` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.628115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bb890-c467-4e72-8c79-f1320440a5bd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:57.629670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bb890-c467-4e72-8c79-f1320440a5bd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:57.650730 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdec9f-f639-4940-94fa-242bc33830fd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:57.685352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d9c55-bf98-4efc-ba5b-3a1c07e7f55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:57.686664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d9c55-bf98-4efc-ba5b-3a1c07e7f55e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.686950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d9c55-bf98-4efc-ba5b-3a1c07e7f55e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.687221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d9c55-bf98-4efc-ba5b-3a1c07e7f55e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.688053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d9c55-bf98-4efc-ba5b-3a1c07e7f55e None admin] RBAC: Adding query filter params (role.id=4f99bdbe513b4500b96f9c530dac52ed, user.id=f1936b1009f242a592450d07c5bc551a, scope.system=all) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:57.688365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d9c55-bf98-4efc-ba5b-3a1c07e7f55e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:57.689825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d9c55-bf98-4efc-ba5b-3a1c07e7f55e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:57.695251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 149/303] 10.222.0.88 () {56 vars in 1308 bytes} [Mon Jul 18 16:29:57 2022] GET /identity/v3/role_assignments?user.id=f1936b1009f242a592450d07c5bc551a&scope.system=all&role.id=4f99bdbe513b4500b96f9c530dac52ed => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:57.697283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ad23c-e2ff-4153-b918-ba3cb7050868 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.697579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ad23c-e2ff-4153-b918-ba3cb7050868 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.697858 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ad23c-e2ff-4153-b918-ba3cb7050868 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.699388 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 150/304] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:57.705991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95765d05-0c19-41a8-a348-9a95c3aa2c6e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.706280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95765d05-0c19-41a8-a348-9a95c3aa2c6e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.706667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95765d05-0c19-41a8-a348-9a95c3aa2c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.713152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 155/305] 10.222.0.88 () {60 vars in 1134 bytes} [Mon Jul 18 16:29:57 2022] POST /identity/v3/groups => generated 229 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:29:57.729699 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:57.734352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-95765d05-0c19-41a8-a348-9a95c3aa2c6e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:57.735782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-95765d05-0c19-41a8-a348-9a95c3aa2c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:57.830810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 151/306] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:57.853224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bb890-c467-4e72-8c79-f1320440a5bd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:57.971945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6704e-0436-475e-aa7f-2bf9cd417b05 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.972322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6704e-0436-475e-aa7f-2bf9cd417b05 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.972512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6704e-0436-475e-aa7f-2bf9cd417b05 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:57.974111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 152/307] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:57.979447 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06279d79-07fb-40a7-9247-b1363b9be32a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:57.979739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06279d79-07fb-40a7-9247-b1363b9be32a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:57.980015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06279d79-07fb-40a7-9247-b1363b9be32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:58.151643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-06279d79-07fb-40a7-9247-b1363b9be32a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:58.152749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-06279d79-07fb-40a7-9247-b1363b9be32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:58.233796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 153/308] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:58.257449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06279d79-07fb-40a7-9247-b1363b9be32a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:58.261641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bb890-c467-4e72-8c79-f1320440a5bd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:58.308918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae5f30-9aa5-41f3-9310-22569202c280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:58.310280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae5f30-9aa5-41f3-9310-22569202c280 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:58.310752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae5f30-9aa5-41f3-9310-22569202c280 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:58.311642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae5f30-9aa5-41f3-9310-22569202c280 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:58.321101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:58.327142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:58.329639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae5f30-9aa5-41f3-9310-22569202c280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-fcae5f30-9aa5-41f3-9310-22569202c280 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:58.343425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:58.344574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:29:58.345655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:29:58.348241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 156/309] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 496 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:29:58.354172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae5f30-9aa5-41f3-9310-22569202c280 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:58.378126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06279d79-07fb-40a7-9247-b1363b9be32a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:58.415835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6457c8-133a-4e0f-99db-c10fc87a40d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:58.416517 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6457c8-133a-4e0f-99db-c10fc87a40d0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:58.416897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6457c8-133a-4e0f-99db-c10fc87a40d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:58.417281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6457c8-133a-4e0f-99db-c10fc87a40d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:58.429727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6457c8-133a-4e0f-99db-c10fc87a40d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-6c6457c8-133a-4e0f-99db-c10fc87a40d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:29:58.437824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:29:58.438723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:29:58.439502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 154/310] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:29:58.446017 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6457c8-133a-4e0f-99db-c10fc87a40d0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:58.479054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae5f30-9aa5-41f3-9310-22569202c280 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:58.518008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26897ae4-683f-4a53-bf4d-65aef3657023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:58.518950 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26897ae4-683f-4a53-bf4d-65aef3657023 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:58.519392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26897ae4-683f-4a53-bf4d-65aef3657023 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:58.519981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26897ae4-683f-4a53-bf4d-65aef3657023 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:58.520951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26897ae4-683f-4a53-bf4d-65aef3657023 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:58.521454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26897ae4-683f-4a53-bf4d-65aef3657023 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:58.523049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26897ae4-683f-4a53-bf4d-65aef3657023 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:58.537125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 157/311] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:29:58.539559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26897ae4-683f-4a53-bf4d-65aef3657023 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:58.578505 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6457c8-133a-4e0f-99db-c10fc87a40d0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:58.587670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:58.603195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:58.616264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631efa4-f2c6-4683-aaba-c3153db90d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:58.616964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631efa4-f2c6-4683-aaba-c3153db90d52 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:58.617252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631efa4-f2c6-4683-aaba-c3153db90d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:58.617524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631efa4-f2c6-4683-aaba-c3153db90d52 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:58.618529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631efa4-f2c6-4683-aaba-c3153db90d52 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:58.618890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631efa4-f2c6-4683-aaba-c3153db90d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:58.620341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631efa4-f2c6-4683-aaba-c3153db90d52 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:58.634741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 155/312] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:29:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:58.636141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631efa4-f2c6-4683-aaba-c3153db90d52 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:58.649168 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26897ae4-683f-4a53-bf4d-65aef3657023 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:58.684698 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cd90f-88b5-4dd7-b62a-e9bf0772f616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:58.686616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd90f-88b5-4dd7-b62a-e9bf0772f616 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:58.687110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd90f-88b5-4dd7-b62a-e9bf0772f616 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:58.687573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd90f-88b5-4dd7-b62a-e9bf0772f616 None admin] PATH_INFO: `/v3/users/c4ce363db1b6428e89ebf4125deb87ab` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:58.689004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cd90f-88b5-4dd7-b62a-e9bf0772f616 None admin] RBAC: Authorizing `identity:get_user(user_id=c4ce363db1b6428e89ebf4125deb87ab)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:58.690770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cd90f-88b5-4dd7-b62a-e9bf0772f616 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:58.697208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 158/313] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:58 2022] GET /identity/v3/users/c4ce363db1b6428e89ebf4125deb87ab => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:58.698287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cd90f-88b5-4dd7-b62a-e9bf0772f616 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:58.747081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631efa4-f2c6-4683-aaba-c3153db90d52 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:58.785797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4236e-2496-46af-aef7-4fae3288283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:58.786677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4236e-2496-46af-aef7-4fae3288283d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:58.786959 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4236e-2496-46af-aef7-4fae3288283d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:58.787230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4236e-2496-46af-aef7-4fae3288283d None admin] PATH_INFO: `/v3/users/0f37a10fe8ea48c68c9e55516732e26c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:58.788518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4236e-2496-46af-aef7-4fae3288283d None admin] RBAC: Authorizing `identity:get_user(user_id=0f37a10fe8ea48c68c9e55516732e26c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:58.790159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4236e-2496-46af-aef7-4fae3288283d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:58.796155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 156/314] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:29:58 2022] GET /identity/v3/users/0f37a10fe8ea48c68c9e55516732e26c => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:58.798923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4236e-2496-46af-aef7-4fae3288283d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:58.807868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cd90f-88b5-4dd7-b62a-e9bf0772f616 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:58.822413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:58.844366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59589c-f767-4973-bc05-43931b6d9fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:58.845717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59589c-f767-4973-bc05-43931b6d9fd1 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:58.846196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59589c-f767-4973-bc05-43931b6d9fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:58.846664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59589c-f767-4973-bc05-43931b6d9fd1 None admin] PATH_INFO: `/v3/system/users/c4ce363db1b6428e89ebf4125deb87ab/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:58.848511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59589c-f767-4973-bc05-43931b6d9fd1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c4ce363db1b6428e89ebf4125deb87ab, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:58.850135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59589c-f767-4973-bc05-43931b6d9fd1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:58.861355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:58.914035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4236e-2496-46af-aef7-4fae3288283d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:58.955957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 159/315] 10.222.0.88 () {58 vars in 1334 bytes} [Mon Jul 18 16:29:58 2022] PUT /identity/v3/system/users/c4ce363db1b6428e89ebf4125deb87ab/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:29:58.966320 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecd80c-d140-4bf3-99a6-f0125756428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:58.967664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecd80c-d140-4bf3-99a6-f0125756428c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:58.967993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecd80c-d140-4bf3-99a6-f0125756428c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:58.968276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecd80c-d140-4bf3-99a6-f0125756428c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:59.050567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecd80c-d140-4bf3-99a6-f0125756428c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:59.052171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecd80c-d140-4bf3-99a6-f0125756428c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:59.110842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:59.168081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 157/316] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:29:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:29:59.173578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59589c-f767-4973-bc05-43931b6d9fd1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:59.283264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59589c-f767-4973-bc05-43931b6d9fd1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:59.321125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dfd87-e31f-48fc-9a35-083b49c75dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:59.322350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dfd87-e31f-48fc-9a35-083b49c75dda None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:59.322823 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dfd87-e31f-48fc-9a35-083b49c75dda None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:59.323295 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dfd87-e31f-48fc-9a35-083b49c75dda None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:59.324763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349dfd87-e31f-48fc-9a35-083b49c75dda None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:59.326983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349dfd87-e31f-48fc-9a35-083b49c75dda None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:59.329309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 160/317] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:29:59 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:59.336928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecd80c-d140-4bf3-99a6-f0125756428c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:59.444661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecd80c-d140-4bf3-99a6-f0125756428c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:29:59.482827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de6628-aa1e-4453-9244-72724e87d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:29:59.483493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de6628-aa1e-4453-9244-72724e87d1ad None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:59.483780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de6628-aa1e-4453-9244-72724e87d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:59.484058 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de6628-aa1e-4453-9244-72724e87d1ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:59.484911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de6628-aa1e-4453-9244-72724e87d1ad None admin] RBAC: Adding query filter params (role.id=4f99bdbe513b4500b96f9c530dac52ed, scope.project.id=c5de3bc9e442415487283bf9414ae97d, user.id=0f37a10fe8ea48c68c9e55516732e26c) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:29:59.485236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de6628-aa1e-4453-9244-72724e87d1ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:29:59.486688 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de6628-aa1e-4453-9244-72724e87d1ad None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:29:59.493667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 158/318] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:29:59 2022] GET /identity/v3/role_assignments?user.id=0f37a10fe8ea48c68c9e55516732e26c&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=4f99bdbe513b4500b96f9c530dac52ed => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:29:59.798400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bb83d-8905-4eda-a000-7fb5eaaddee9 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:59.798783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bb83d-8905-4eda-a000-7fb5eaaddee9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:59.798979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bb83d-8905-4eda-a000-7fb5eaaddee9 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:59.800390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 161/319] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:29:59.805787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711c987-3a92-4b95-a3af-f59145a19a27 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:59.809048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711c987-3a92-4b95-a3af-f59145a19a27 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:59.809048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711c987-3a92-4b95-a3af-f59145a19a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:59.823352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046d81-7cb0-45e2-bd12-ccfecdc122fd None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:59.823635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046d81-7cb0-45e2-bd12-ccfecdc122fd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:59.823912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046d81-7cb0-45e2-bd12-ccfecdc122fd None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:59.824757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0711c987-3a92-4b95-a3af-f59145a19a27 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:59.825525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 162/320] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:29:59.826016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0711c987-3a92-4b95-a3af-f59145a19a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:59.831920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9060c-172e-4bf9-93f9-bba160edf85d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:29:59.832212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9060c-172e-4bf9-93f9-bba160edf85d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:29:59.832494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9060c-172e-4bf9-93f9-bba160edf85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:29:59.838375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:59.850970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8cc9060c-172e-4bf9-93f9-bba160edf85d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:29:59.854583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8cc9060c-172e-4bf9-93f9-bba160edf85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:29:59.895555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:29:59.896327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 159/321] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:29:59.918148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711c987-3a92-4b95-a3af-f59145a19a27 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:29:59.924355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 163/322] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:29:59.946985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9060c-172e-4bf9-93f9-bba160edf85d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:00.032533 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711c987-3a92-4b95-a3af-f59145a19a27 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:00.059375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9060c-172e-4bf9-93f9-bba160edf85d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:00.074003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d12b8a-a33a-4326-ae9b-3c5a09f42ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:00.074813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d12b8a-a33a-4326-ae9b-3c5a09f42ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:00.075098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d12b8a-a33a-4326-ae9b-3c5a09f42ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:00.075372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d12b8a-a33a-4326-ae9b-3c5a09f42ef4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:00.086237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d12b8a-a33a-4326-ae9b-3c5a09f42ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d6d12b8a-a33a-4326-ae9b-3c5a09f42ef4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:00.094079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:00.095007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:00.095795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 160/323] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:29:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:00.101053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d12b8a-a33a-4326-ae9b-3c5a09f42ef4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:00.106973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdbf18-a2fb-4e24-a6be-f85bd0e9f983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:00.108308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdbf18-a2fb-4e24-a6be-f85bd0e9f983 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:00.108590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdbf18-a2fb-4e24-a6be-f85bd0e9f983 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:00.108861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdbf18-a2fb-4e24-a6be-f85bd0e9f983 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:00.110085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdbf18-a2fb-4e24-a6be-f85bd0e9f983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:00.111572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdbf18-a2fb-4e24-a6be-f85bd0e9f983 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:00.111756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:00.113816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 164/324] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:29:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:00.119450 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdbf18-a2fb-4e24-a6be-f85bd0e9f983 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:00.175297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:00.207823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d12b8a-a33a-4326-ae9b-3c5a09f42ef4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:00.238126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdbf18-a2fb-4e24-a6be-f85bd0e9f983 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:00.244632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd5563-10c5-4f7c-9426-9c86751d603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:00.245339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd5563-10c5-4f7c-9426-9c86751d603a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:00.245627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd5563-10c5-4f7c-9426-9c86751d603a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:00.245906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd5563-10c5-4f7c-9426-9c86751d603a None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:00.246792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd5563-10c5-4f7c-9426-9c86751d603a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:00.247104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd5563-10c5-4f7c-9426-9c86751d603a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:00.249067 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd5563-10c5-4f7c-9426-9c86751d603a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:00.262754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 161/325] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:30:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:00.264205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd5563-10c5-4f7c-9426-9c86751d603a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:00.281609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d6b6b-67fd-459e-be0d-be8de3a656b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:00.282538 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6b6b-67fd-459e-be0d-be8de3a656b7 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:00.282822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6b6b-67fd-459e-be0d-be8de3a656b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:00.283101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d6b6b-67fd-459e-be0d-be8de3a656b7 None admin] PATH_INFO: `/v3/groups` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:00.283802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d6b6b-67fd-459e-be0d-be8de3a656b7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:00.285375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d6b6b-67fd-459e-be0d-be8de3a656b7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:00.335812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:00.372201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd5563-10c5-4f7c-9426-9c86751d603a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:00.389390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 165/326] 10.222.0.88 () {60 vars in 1134 bytes} [Mon Jul 18 16:30:00 2022] POST /identity/v3/groups => generated 226 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:30:00.419903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1a7b3-e395-45dd-b6ca-81fa75b09092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:00.420824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1a7b3-e395-45dd-b6ca-81fa75b09092 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:00.421138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1a7b3-e395-45dd-b6ca-81fa75b09092 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:00.421412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1a7b3-e395-45dd-b6ca-81fa75b09092 None admin] PATH_INFO: `/v3/users/f1936b1009f242a592450d07c5bc551a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:00.422673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1a7b3-e395-45dd-b6ca-81fa75b09092 None admin] RBAC: Authorizing `identity:get_user(user_id=f1936b1009f242a592450d07c5bc551a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:00.424126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1a7b3-e395-45dd-b6ca-81fa75b09092 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:00.425842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 162/327] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:30:00 2022] GET /identity/v3/users/f1936b1009f242a592450d07c5bc551a => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:00.432077 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d6b6b-67fd-459e-be0d-be8de3a656b7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:00.548929 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d6b6b-67fd-459e-be0d-be8de3a656b7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:00.550578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:00.585939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77666395-abe6-47e8-80c4-be5d7ffdef5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:00.586740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77666395-abe6-47e8-80c4-be5d7ffdef5d None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:00.587035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77666395-abe6-47e8-80c4-be5d7ffdef5d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:00.587309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77666395-abe6-47e8-80c4-be5d7ffdef5d None admin] PATH_INFO: `/v3/system/users/f1936b1009f242a592450d07c5bc551a/roles/4f99bdbe513b4500b96f9c530dac52ed` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:00.589001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77666395-abe6-47e8-80c4-be5d7ffdef5d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f1936b1009f242a592450d07c5bc551a, role_id=4f99bdbe513b4500b96f9c530dac52ed)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:00.590443 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77666395-abe6-47e8-80c4-be5d7ffdef5d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:00.672814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 166/328] 10.222.0.88 () {58 vars in 1334 bytes} [Mon Jul 18 16:30:00 2022] PUT /identity/v3/system/users/f1936b1009f242a592450d07c5bc551a/roles/4f99bdbe513b4500b96f9c530dac52ed => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:30:01.109929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c21073-c41f-4810-92b6-c0103bb46589 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:01.110917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c21073-c41f-4810-92b6-c0103bb46589 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:01.111418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c21073-c41f-4810-92b6-c0103bb46589 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:01.113332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 163/329] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:01.118871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdd8d3-8806-4bd4-9228-9c8f38f286f9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:01.119167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdd8d3-8806-4bd4-9228-9c8f38f286f9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:01.119453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdd8d3-8806-4bd4-9228-9c8f38f286f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:01.142705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6fbdd8d3-8806-4bd4-9228-9c8f38f286f9 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:01.143906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6fbdd8d3-8806-4bd4-9228-9c8f38f286f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:01.234822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 167/330] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:01.261795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c21073-c41f-4810-92b6-c0103bb46589 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:01.401661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c21073-c41f-4810-92b6-c0103bb46589 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:01.452567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf93e38-82d0-4a07-bbae-c49dcac5d80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:01.453429 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf93e38-82d0-4a07-bbae-c49dcac5d80b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:01.453712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf93e38-82d0-4a07-bbae-c49dcac5d80b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:01.453985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf93e38-82d0-4a07-bbae-c49dcac5d80b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:01.457176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:01.471337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf93e38-82d0-4a07-bbae-c49dcac5d80b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-0cf93e38-82d0-4a07-bbae-c49dcac5d80b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:01.479133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:01.479999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:01.481211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 164/331] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:01.494385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdd8d3-8806-4bd4-9228-9c8f38f286f9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:01.651776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:01.658972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdd8d3-8806-4bd4-9228-9c8f38f286f9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:01.703313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efdf65-3867-49d3-86bd-4d2ca5a31a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:01.703996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efdf65-3867-49d3-86bd-4d2ca5a31a15 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:01.704288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efdf65-3867-49d3-86bd-4d2ca5a31a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:01.704564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efdf65-3867-49d3-86bd-4d2ca5a31a15 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:01.705352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efdf65-3867-49d3-86bd-4d2ca5a31a15 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:01.705661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efdf65-3867-49d3-86bd-4d2ca5a31a15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:01.707073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efdf65-3867-49d3-86bd-4d2ca5a31a15 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:01.722238 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 168/332] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:30:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:01.728328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf93e38-82d0-4a07-bbae-c49dcac5d80b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:01.878898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf93e38-82d0-4a07-bbae-c49dcac5d80b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:01.945660 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54a15a-a048-404c-9155-2cc18e77f49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:01.946581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a15a-a048-404c-9155-2cc18e77f49f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:01.946903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a15a-a048-404c-9155-2cc18e77f49f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:01.947252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54a15a-a048-404c-9155-2cc18e77f49f None admin] PATH_INFO: `/v3/users/c4ce363db1b6428e89ebf4125deb87ab` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:01.948743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54a15a-a048-404c-9155-2cc18e77f49f None admin] RBAC: Authorizing `identity:get_user(user_id=c4ce363db1b6428e89ebf4125deb87ab)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:01.950287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54a15a-a048-404c-9155-2cc18e77f49f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:01.952346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 165/333] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:30:01 2022] GET /identity/v3/users/c4ce363db1b6428e89ebf4125deb87ab => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:01.957969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efdf65-3867-49d3-86bd-4d2ca5a31a15 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:02.101022 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efdf65-3867-49d3-86bd-4d2ca5a31a15 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:02.150890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae725ca-f774-4232-a784-01fe6c078ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:02.151477 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae725ca-f774-4232-a784-01fe6c078ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:02.151765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae725ca-f774-4232-a784-01fe6c078ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:02.152040 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae725ca-f774-4232-a784-01fe6c078ac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:02.152908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae725ca-f774-4232-a784-01fe6c078ac7 None admin] RBAC: Adding query filter params (role.id=3b2e1fad42f441e9a61c142477a50b11, user.id=c4ce363db1b6428e89ebf4125deb87ab, scope.system=all) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:02.153221 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae725ca-f774-4232-a784-01fe6c078ac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:02.154732 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae725ca-f774-4232-a784-01fe6c078ac7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:02.172553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 169/334] 10.222.0.88 () {56 vars in 1308 bytes} [Mon Jul 18 16:30:01 2022] GET /identity/v3/role_assignments?user.id=c4ce363db1b6428e89ebf4125deb87ab&scope.system=all&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:02.934100 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a01703-e5a7-4885-aa75-830edd76d354 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:02.935358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a01703-e5a7-4885-aa75-830edd76d354 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:02.936370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a01703-e5a7-4885-aa75-830edd76d354 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:02.938334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 166/335] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:02.943525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550636ec-7eb8-456f-ab8c-982f85dd028a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:02.944407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550636ec-7eb8-456f-ab8c-982f85dd028a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:02.944905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550636ec-7eb8-456f-ab8c-982f85dd028a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:02.969402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-550636ec-7eb8-456f-ab8c-982f85dd028a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:02.971029 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-550636ec-7eb8-456f-ab8c-982f85dd028a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:02.983562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:03.057326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 170/336] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:03.097579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a01703-e5a7-4885-aa75-830edd76d354 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:03.172842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:03.221651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a01703-e5a7-4885-aa75-830edd76d354 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:03.265433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f330c-cf21-41dc-af80-c07c8f941895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:03.266295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f330c-cf21-41dc-af80-c07c8f941895 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:03.266583 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f330c-cf21-41dc-af80-c07c8f941895 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:03.266860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f330c-cf21-41dc-af80-c07c8f941895 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:03.281557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f330c-cf21-41dc-af80-c07c8f941895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-412f330c-cf21-41dc-af80-c07c8f941895 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:03.289717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:03.290637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:03.292040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 167/337] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:03.297329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550636ec-7eb8-456f-ab8c-982f85dd028a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:03.408835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550636ec-7eb8-456f-ab8c-982f85dd028a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:03.459632 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654e770-5201-4ea6-a799-d81541077bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:03.460496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654e770-5201-4ea6-a799-d81541077bcd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:03.460778 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654e770-5201-4ea6-a799-d81541077bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:03.461080 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654e770-5201-4ea6-a799-d81541077bcd None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:03.461862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654e770-5201-4ea6-a799-d81541077bcd None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:03.462167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654e770-5201-4ea6-a799-d81541077bcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:03.463595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654e770-5201-4ea6-a799-d81541077bcd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:03.473823 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 171/338] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:30:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:03.479675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f330c-cf21-41dc-af80-c07c8f941895 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:03.618721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f330c-cf21-41dc-af80-c07c8f941895 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:03.648369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:03.659806 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afbe6f-b694-425e-b02e-a9e99d22a089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:03.660532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afbe6f-b694-425e-b02e-a9e99d22a089 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:03.660824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afbe6f-b694-425e-b02e-a9e99d22a089 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:03.661189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24afbe6f-b694-425e-b02e-a9e99d22a089 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:03.699948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced8e1-a810-4648-8105-a5ef6e9d797f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:03.700280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced8e1-a810-4648-8105-a5ef6e9d797f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:03.700511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ced8e1-a810-4648-8105-a5ef6e9d797f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:03.702047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 172/339] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:03.709454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d28ea-4963-46fb-ba9d-f6913bd846e9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:03.709744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d28ea-4963-46fb-ba9d-f6913bd846e9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:03.710023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d28ea-4963-46fb-ba9d-f6913bd846e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:03.742776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-540d28ea-4963-46fb-ba9d-f6913bd846e9 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:03.744239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-540d28ea-4963-46fb-ba9d-f6913bd846e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:03.747523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:03.765911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afbe6f-b694-425e-b02e-a9e99d22a089 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:03.767426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24afbe6f-b694-425e-b02e-a9e99d22a089 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:03.879405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 173/340] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:03.880762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa205068-6ce5-47bf-a1e3-1a9e89ff6a73 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:03.881104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa205068-6ce5-47bf-a1e3-1a9e89ff6a73 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:03.881391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa205068-6ce5-47bf-a1e3-1a9e89ff6a73 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:03.882979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 174/341] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:03.888580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9ce0-ed8a-4aea-b728-c5242176acfd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:03.889008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9ce0-ed8a-4aea-b728-c5242176acfd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:03.889358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9ce0-ed8a-4aea-b728-c5242176acfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:03.918854 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2a3e9ce0-ed8a-4aea-b728-c5242176acfd None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:03.920612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2a3e9ce0-ed8a-4aea-b728-c5242176acfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:03.957942 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 168/342] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:03.958893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afbe6f-b694-425e-b02e-a9e99d22a089 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:04.007648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:04.008561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 175/343] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:30:04.010107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9ce0-ed8a-4aea-b728-c5242176acfd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:04.084783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:04.126350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9ce0-ed8a-4aea-b728-c5242176acfd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:04.153438 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24afbe6f-b694-425e-b02e-a9e99d22a089 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:04.171301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc4dcf-b33c-4739-8ad4-d71e344c8ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:04.173139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4dcf-b33c-4739-8ad4-d71e344c8ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.173766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4dcf-b33c-4739-8ad4-d71e344c8ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.174361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc4dcf-b33c-4739-8ad4-d71e344c8ad1 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.176594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc4dcf-b33c-4739-8ad4-d71e344c8ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:04.178304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc4dcf-b33c-4739-8ad4-d71e344c8ad1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:04.180568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 176/344] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:04 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:04.182195 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc4dcf-b33c-4739-8ad4-d71e344c8ad1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:04.198071 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1413d5-1543-4d26-884c-65bc83d35cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:04.199314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1413d5-1543-4d26-884c-65bc83d35cec None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.199314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1413d5-1543-4d26-884c-65bc83d35cec None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.199678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1413d5-1543-4d26-884c-65bc83d35cec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.212348 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1413d5-1543-4d26-884c-65bc83d35cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-fa1413d5-1543-4d26-884c-65bc83d35cec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:04.220674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:04.221832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 16:30:04.222653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:04.223247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 169/345] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:04.229924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1413d5-1543-4d26-884c-65bc83d35cec None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:04.362663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc4dcf-b33c-4739-8ad4-d71e344c8ad1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:04.379555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:04.392467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1413d5-1543-4d26-884c-65bc83d35cec None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:04.438838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322141c2-88e1-414a-a0fd-da1a6291bc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:04.439949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322141c2-88e1-414a-a0fd-da1a6291bc5b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.440379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322141c2-88e1-414a-a0fd-da1a6291bc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.441004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322141c2-88e1-414a-a0fd-da1a6291bc5b None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.450841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322141c2-88e1-414a-a0fd-da1a6291bc5b None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:04.452702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322141c2-88e1-414a-a0fd-da1a6291bc5b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:04.455061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 170/346] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:04 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:04.455984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322141c2-88e1-414a-a0fd-da1a6291bc5b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:04.456517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c097c10-03c0-44b1-9479-62500cda4992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:04.457508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c097c10-03c0-44b1-9479-62500cda4992 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.457887 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c097c10-03c0-44b1-9479-62500cda4992 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.458259 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c097c10-03c0-44b1-9479-62500cda4992 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.473619 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:04.494834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c097c10-03c0-44b1-9479-62500cda4992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8c097c10-03c0-44b1-9479-62500cda4992 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:04.507490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:04.508647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:04.509695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:04.515975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 177/347] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:30:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:04.515975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c097c10-03c0-44b1-9479-62500cda4992 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:04.592804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322141c2-88e1-414a-a0fd-da1a6291bc5b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:04.637016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c097c10-03c0-44b1-9479-62500cda4992 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:04.638285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ed499-f4c2-4a9e-8836-3b9907fa0b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:04.639167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ed499-f4c2-4a9e-8836-3b9907fa0b59 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.639478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ed499-f4c2-4a9e-8836-3b9907fa0b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.639760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ed499-f4c2-4a9e-8836-3b9907fa0b59 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.641148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ed499-f4c2-4a9e-8836-3b9907fa0b59 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:04.645296 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ed499-f4c2-4a9e-8836-3b9907fa0b59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:04.647270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ed499-f4c2-4a9e-8836-3b9907fa0b59 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:04.658973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 171/348] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:30:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:04.660448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b41af-6fae-4c53-baad-4750db36c75b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.661776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b41af-6fae-4c53-baad-4750db36c75b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.661776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b41af-6fae-4c53-baad-4750db36c75b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.664918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 172/349] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:04.664918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b41af-6fae-4c53-baad-4750db36c75b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:04.686361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:04.696229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168cf96-3cdc-43c6-9cda-5aead2e8b838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:04.697358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168cf96-3cdc-43c6-9cda-5aead2e8b838 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.697809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168cf96-3cdc-43c6-9cda-5aead2e8b838 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.707325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168cf96-3cdc-43c6-9cda-5aead2e8b838 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.707325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168cf96-3cdc-43c6-9cda-5aead2e8b838 None admin] RBAC: Adding query filter params (group.id=e0162af404b44cd5b4d8886df243dac7, role.id=3b2e1fad42f441e9a61c142477a50b11, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:04.707325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168cf96-3cdc-43c6-9cda-5aead2e8b838 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:04.707325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168cf96-3cdc-43c6-9cda-5aead2e8b838 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:04.708123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 178/350] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Jul 18 16:30:04 2022] GET /identity/v3/role_assignments?group.id=e0162af404b44cd5b4d8886df243dac7&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:04.709213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168cf96-3cdc-43c6-9cda-5aead2e8b838 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:04.783065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:04.846498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168cf96-3cdc-43c6-9cda-5aead2e8b838 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:04.847844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b41af-6fae-4c53-baad-4750db36c75b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:04.899136 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b311c-403f-4918-a696-cfd38d7bc4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:04.899136 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b311c-403f-4918-a696-cfd38d7bc4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.899407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b311c-403f-4918-a696-cfd38d7bc4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.899740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b311c-403f-4918-a696-cfd38d7bc4e5 None admin] PATH_INFO: `/v3/users/f1936b1009f242a592450d07c5bc551a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.901543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b311c-403f-4918-a696-cfd38d7bc4e5 None admin] RBAC: Authorizing `identity:get_user(user_id=f1936b1009f242a592450d07c5bc551a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:04.903009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b311c-403f-4918-a696-cfd38d7bc4e5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:04.905289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 179/351] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:30:04 2022] GET /identity/v3/users/f1936b1009f242a592450d07c5bc551a => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:04.907036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a009b-0e4e-435d-829e-299757ffa2d5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.907326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a009b-0e4e-435d-829e-299757ffa2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.907593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa4844-d52f-48ba-84e9-8dbfc075d84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:04.907710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a009b-0e4e-435d-829e-299757ffa2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.908343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa4844-d52f-48ba-84e9-8dbfc075d84d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.908686 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa4844-d52f-48ba-84e9-8dbfc075d84d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.909005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa4844-d52f-48ba-84e9-8dbfc075d84d None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.909780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa4844-d52f-48ba-84e9-8dbfc075d84d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:04.910156 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa4844-d52f-48ba-84e9-8dbfc075d84d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:04.911595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa4844-d52f-48ba-84e9-8dbfc075d84d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:04.923511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 173/352] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:04.925007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119c8ab-a1b1-4092-9f7d-aa52bbc1e83e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:04.925305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119c8ab-a1b1-4092-9f7d-aa52bbc1e83e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:04.925642 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119c8ab-a1b1-4092-9f7d-aa52bbc1e83e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:04.927207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 174/353] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:04.927940 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119c8ab-a1b1-4092-9f7d-aa52bbc1e83e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:04.936908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cb8a009b-0e4e-435d-829e-299757ffa2d5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:04.936908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cb8a009b-0e4e-435d-829e-299757ffa2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:05.031598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 180/354] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:05.032494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a009b-0e4e-435d-829e-299757ffa2d5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:05.039979 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:05.094659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119c8ab-a1b1-4092-9f7d-aa52bbc1e83e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:05.137522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:05.156604 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c73d96-208e-4843-8351-06674b0d8625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:05.157224 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c73d96-208e-4843-8351-06674b0d8625 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:05.157512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c73d96-208e-4843-8351-06674b0d8625 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:05.157789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c73d96-208e-4843-8351-06674b0d8625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:05.158640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c73d96-208e-4843-8351-06674b0d8625 None admin] RBAC: Adding query filter params (role.id=4f99bdbe513b4500b96f9c530dac52ed, user.id=f1936b1009f242a592450d07c5bc551a, scope.system=all) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:05.158952 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c73d96-208e-4843-8351-06674b0d8625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:05.160429 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c73d96-208e-4843-8351-06674b0d8625 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:05.173391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 175/355] 10.222.0.88 () {56 vars in 1308 bytes} [Mon Jul 18 16:30:04 2022] GET /identity/v3/role_assignments?user.id=f1936b1009f242a592450d07c5bc551a&scope.system=all&role.id=4f99bdbe513b4500b96f9c530dac52ed => generated 510 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:05.174698 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166d31-ae78-43b1-839f-503c445816f5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:05.174981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166d31-ae78-43b1-839f-503c445816f5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:05.175263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6166d31-ae78-43b1-839f-503c445816f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:05.190883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a009b-0e4e-435d-829e-299757ffa2d5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:05.198666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e6166d31-ae78-43b1-839f-503c445816f5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:05.200475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e6166d31-ae78-43b1-839f-503c445816f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:05.249739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a00011-5d46-467f-a845-dc15d9f4c689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:05.250418 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a00011-5d46-467f-a845-dc15d9f4c689 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:05.250736 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a00011-5d46-467f-a845-dc15d9f4c689 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:05.250980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a00011-5d46-467f-a845-dc15d9f4c689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:05.295782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 176/356] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:05.297090 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6166d31-ae78-43b1-839f-503c445816f5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:05.359425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a00011-5d46-467f-a845-dc15d9f4c689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:05.360974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a00011-5d46-467f-a845-dc15d9f4c689 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:05.367077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:05.414397 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6166d31-ae78-43b1-839f-503c445816f5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:05.446387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:05.454241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834804c-27d2-4bda-866e-f40395208ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:05.455057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834804c-27d2-4bda-866e-f40395208ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:05.455629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834804c-27d2-4bda-866e-f40395208ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:05.455629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834804c-27d2-4bda-866e-f40395208ef0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:05.472583 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834804c-27d2-4bda-866e-f40395208ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1834804c-27d2-4bda-866e-f40395208ef0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:05.482912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:05.483852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:05.484678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:05.487407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 177/357] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:05.488418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834804c-27d2-4bda-866e-f40395208ef0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:05.531676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 181/358] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:05.533954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29ef53-f53b-4ae7-88b8-0ebfbd0f93a2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:05.533954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29ef53-f53b-4ae7-88b8-0ebfbd0f93a2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:05.533954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29ef53-f53b-4ae7-88b8-0ebfbd0f93a2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:05.536960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 182/359] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:05.537773 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29ef53-f53b-4ae7-88b8-0ebfbd0f93a2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:05.611894 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834804c-27d2-4bda-866e-f40395208ef0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:05.640191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:05.662369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60382286-d793-4e3f-bfab-104b19864e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:05.663193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60382286-d793-4e3f-bfab-104b19864e0e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:05.663477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60382286-d793-4e3f-bfab-104b19864e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:05.663767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60382286-d793-4e3f-bfab-104b19864e0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:05.673078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29ef53-f53b-4ae7-88b8-0ebfbd0f93a2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:05.678733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60382286-d793-4e3f-bfab-104b19864e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-60382286-d793-4e3f-bfab-104b19864e0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:05.687357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:05.688239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:05.689046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:05.690694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 178/360] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:30:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:05.694069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60382286-d793-4e3f-bfab-104b19864e0e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:05.721235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faea32-89ae-459e-9f37-b1a64ed36105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:05.722061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faea32-89ae-459e-9f37-b1a64ed36105 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:05.722496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faea32-89ae-459e-9f37-b1a64ed36105 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:05.722982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faea32-89ae-459e-9f37-b1a64ed36105 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:05.724180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8faea32-89ae-459e-9f37-b1a64ed36105 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:05.724180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8faea32-89ae-459e-9f37-b1a64ed36105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:05.725565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8faea32-89ae-459e-9f37-b1a64ed36105 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:05.735331 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 183/361] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:30:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:05.737005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784856-adff-45d5-9683-207d470b06cd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:05.737503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784856-adff-45d5-9683-207d470b06cd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:05.738017 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784856-adff-45d5-9683-207d470b06cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:05.757416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-27784856-adff-45d5-9683-207d470b06cd None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:05.758949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-27784856-adff-45d5-9683-207d470b06cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:05.780244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:05.829123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60382286-d793-4e3f-bfab-104b19864e0e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:05.847268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 184/362] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:05.848222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27784856-adff-45d5-9683-207d470b06cd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:05.867446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da989b5d-ffdc-43ae-9a16-5a74f50a95f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:05.870049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da989b5d-ffdc-43ae-9a16-5a74f50a95f6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:05.870330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da989b5d-ffdc-43ae-9a16-5a74f50a95f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:05.870984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da989b5d-ffdc-43ae-9a16-5a74f50a95f6 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:05.872333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da989b5d-ffdc-43ae-9a16-5a74f50a95f6 None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:05.875387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da989b5d-ffdc-43ae-9a16-5a74f50a95f6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:05.881043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 179/363] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:05 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:05.882027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da989b5d-ffdc-43ae-9a16-5a74f50a95f6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:05.955539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:05.983152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27784856-adff-45d5-9683-207d470b06cd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:06.022565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da989b5d-ffdc-43ae-9a16-5a74f50a95f6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:06.040515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693b94d-af50-4ff2-9334-989eeb9219c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:06.042339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693b94d-af50-4ff2-9334-989eeb9219c3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:06.042909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693b94d-af50-4ff2-9334-989eeb9219c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:06.043203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693b94d-af50-4ff2-9334-989eeb9219c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:06.043983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693b94d-af50-4ff2-9334-989eeb9219c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:06.044291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693b94d-af50-4ff2-9334-989eeb9219c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:06.046427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693b94d-af50-4ff2-9334-989eeb9219c3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:06.056110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 185/364] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:06.056994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693b94d-af50-4ff2-9334-989eeb9219c3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:06.070678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b05a25-880f-476f-9a5d-35181e1d0afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:06.072611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b05a25-880f-476f-9a5d-35181e1d0afc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:06.073945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b05a25-880f-476f-9a5d-35181e1d0afc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:06.074416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b05a25-880f-476f-9a5d-35181e1d0afc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:06.076933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:06.166283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b05a25-880f-476f-9a5d-35181e1d0afc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:06.167836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b05a25-880f-476f-9a5d-35181e1d0afc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:06.210427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693b94d-af50-4ff2-9334-989eeb9219c3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:06.236338 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:06.258992 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd1e3e-7b31-44f0-a5a2-0561c93e5ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:06.260318 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd1e3e-7b31-44f0-a5a2-0561c93e5ddc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:06.260604 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd1e3e-7b31-44f0-a5a2-0561c93e5ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:06.260934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd1e3e-7b31-44f0-a5a2-0561c93e5ddc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:06.278218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd1e3e-7b31-44f0-a5a2-0561c93e5ddc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-19bd1e3e-7b31-44f0-a5a2-0561c93e5ddc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:06.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:06.288449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:06.289294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:06.289916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 186/365] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:06.290818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd1e3e-7b31-44f0-a5a2-0561c93e5ddc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:06.310453 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 180/366] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:06.311318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b05a25-880f-476f-9a5d-35181e1d0afc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:06.377662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:06.417033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd1e3e-7b31-44f0-a5a2-0561c93e5ddc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:06.431852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b05a25-880f-476f-9a5d-35181e1d0afc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:06.470317 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de806b-9d8d-4f65-891f-a646d2aff5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:06.471111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de806b-9d8d-4f65-891f-a646d2aff5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:06.471400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de806b-9d8d-4f65-891f-a646d2aff5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:06.471675 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de806b-9d8d-4f65-891f-a646d2aff5d2 None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:06.473473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de806b-9d8d-4f65-891f-a646d2aff5d2 None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:06.474950 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de806b-9d8d-4f65-891f-a646d2aff5d2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:06.476591 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 187/367] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:06 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:06.477390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de806b-9d8d-4f65-891f-a646d2aff5d2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:06.484415 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e103b0-1eb8-42c9-bfc9-5c5171a1b98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:06.485114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e103b0-1eb8-42c9-bfc9-5c5171a1b98e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:06.485398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e103b0-1eb8-42c9-bfc9-5c5171a1b98e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:06.485672 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e103b0-1eb8-42c9-bfc9-5c5171a1b98e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:06.506308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:06.572322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e103b0-1eb8-42c9-bfc9-5c5171a1b98e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:06.573837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e103b0-1eb8-42c9-bfc9-5c5171a1b98e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:06.614910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de806b-9d8d-4f65-891f-a646d2aff5d2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:06.671247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252e2c9-cea2-4546-8d90-bf5277973221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:06.671937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252e2c9-cea2-4546-8d90-bf5277973221 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:06.672224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252e2c9-cea2-4546-8d90-bf5277973221 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:06.672503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252e2c9-cea2-4546-8d90-bf5277973221 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:06.673924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252e2c9-cea2-4546-8d90-bf5277973221 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:06.673924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252e2c9-cea2-4546-8d90-bf5277973221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:06.675784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252e2c9-cea2-4546-8d90-bf5277973221 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:06.684208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 188/368] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:06.686327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252e2c9-cea2-4546-8d90-bf5277973221 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:06.689251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:06.704931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 181/369] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:06.707726 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e103b0-1eb8-42c9-bfc9-5c5171a1b98e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:06.770429 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:06.821119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252e2c9-cea2-4546-8d90-bf5277973221 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:06.852151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e103b0-1eb8-42c9-bfc9-5c5171a1b98e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:06.882888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba54fb-8fe3-4d9e-94ff-e33f283f4d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:06.883717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba54fb-8fe3-4d9e-94ff-e33f283f4d43 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:06.884002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba54fb-8fe3-4d9e-94ff-e33f283f4d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:06.884282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba54fb-8fe3-4d9e-94ff-e33f283f4d43 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:06.885597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba54fb-8fe3-4d9e-94ff-e33f283f4d43 None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:06.887279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba54fb-8fe3-4d9e-94ff-e33f283f4d43 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:06.890071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 189/370] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:06 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:06.890695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba54fb-8fe3-4d9e-94ff-e33f283f4d43 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:06.908326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0436a7-d1e1-44b6-a744-a8dc24f2b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:06.909094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0436a7-d1e1-44b6-a744-a8dc24f2b439 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:06.909378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0436a7-d1e1-44b6-a744-a8dc24f2b439 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:06.909648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0436a7-d1e1-44b6-a744-a8dc24f2b439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:06.910496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0436a7-d1e1-44b6-a744-a8dc24f2b439 None admin] RBAC: Adding query filter params (group.id=e0162af404b44cd5b4d8886df243dac7, role.id=c465d586f0ec4b59815b23087d757fd2, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:06.910808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0436a7-d1e1-44b6-a744-a8dc24f2b439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:06.912270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0436a7-d1e1-44b6-a744-a8dc24f2b439 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:06.920641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 182/371] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Jul 18 16:30:06 2022] GET /identity/v3/role_assignments?group.id=e0162af404b44cd5b4d8886df243dac7&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=c465d586f0ec4b59815b23087d757fd2 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:06.920641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0436a7-d1e1-44b6-a744-a8dc24f2b439 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:07.017082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:07.080450 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba54fb-8fe3-4d9e-94ff-e33f283f4d43 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:07.082055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0436a7-d1e1-44b6-a744-a8dc24f2b439 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:07.101595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:07.130154 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8c9c8-448d-41ae-8833-bee3715c99fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:07.130849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8c9c8-448d-41ae-8833-bee3715c99fb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.131144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8c9c8-448d-41ae-8833-bee3715c99fb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.131425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8c9c8-448d-41ae-8833-bee3715c99fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:07.135380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94422b-64aa-46b9-aede-18bf7a997c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:07.136551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94422b-64aa-46b9-aede-18bf7a997c1b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.137066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94422b-64aa-46b9-aede-18bf7a997c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.137483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94422b-64aa-46b9-aede-18bf7a997c1b None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:07.138929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94422b-64aa-46b9-aede-18bf7a997c1b None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:07.140541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94422b-64aa-46b9-aede-18bf7a997c1b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:07.143122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 183/372] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:06 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:07.144104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94422b-64aa-46b9-aede-18bf7a997c1b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:07.233803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8c9c8-448d-41ae-8833-bee3715c99fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:07.235453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8c9c8-448d-41ae-8833-bee3715c99fb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:07.290518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94422b-64aa-46b9-aede-18bf7a997c1b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:07.339300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702af5b-bde0-44a3-acc0-b7015dc25293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:07.340123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702af5b-bde0-44a3-acc0-b7015dc25293 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.340413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702af5b-bde0-44a3-acc0-b7015dc25293 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.340697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702af5b-bde0-44a3-acc0-b7015dc25293 None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:07.342633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702af5b-bde0-44a3-acc0-b7015dc25293 None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:07.344152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702af5b-bde0-44a3-acc0-b7015dc25293 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:07.346167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 184/373] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:07 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:07.346801 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:07.347456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6018-4f09-4c6b-831a-2b41f02faf04 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.347691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6018-4f09-4c6b-831a-2b41f02faf04 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.347972 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6018-4f09-4c6b-831a-2b41f02faf04 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:07.350111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 185/374] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:07.351243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c6018-4f09-4c6b-831a-2b41f02faf04 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:07.392394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 190/375] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:07.393306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8c9c8-448d-41ae-8833-bee3715c99fb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:07.417353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:07.512452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c6018-4f09-4c6b-831a-2b41f02faf04 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:07.523519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8c9c8-448d-41ae-8833-bee3715c99fb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:07.564087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d5250-048d-41e6-9fb7-2f485d0f8e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:07.565235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d5250-048d-41e6-9fb7-2f485d0f8e1b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.565519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d5250-048d-41e6-9fb7-2f485d0f8e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.565792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d5250-048d-41e6-9fb7-2f485d0f8e1b None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:07.566978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d5250-048d-41e6-9fb7-2f485d0f8e1b None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:07.568426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d5250-048d-41e6-9fb7-2f485d0f8e1b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:07.570411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 186/376] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:07 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:07.571597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff44c9-611c-4973-86f7-075caafdc833 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.571884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff44c9-611c-4973-86f7-075caafdc833 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.572262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff44c9-611c-4973-86f7-075caafdc833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:07.579398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8d083-2cd5-4771-a1bf-cbd42599b0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:07.580066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8d083-2cd5-4771-a1bf-cbd42599b0fb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.580351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8d083-2cd5-4771-a1bf-cbd42599b0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.580638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8d083-2cd5-4771-a1bf-cbd42599b0fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:07.581541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8d083-2cd5-4771-a1bf-cbd42599b0fb None admin] RBAC: Adding query filter params (group.id=e0162af404b44cd5b4d8886df243dac7, role.id=3b2e1fad42f441e9a61c142477a50b11, scope.project.id=c5de3bc9e442415487283bf9414ae97d) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:07.581853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8d083-2cd5-4771-a1bf-cbd42599b0fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:07.583334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8d083-2cd5-4771-a1bf-cbd42599b0fb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:07.589584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-01ff44c9-611c-4973-86f7-075caafdc833 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:07.590735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-01ff44c9-611c-4973-86f7-075caafdc833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:07.596973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 191/377] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Jul 18 16:30:07 2022] GET /identity/v3/role_assignments?group.id=e0162af404b44cd5b4d8886df243dac7&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:07.598016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8d083-2cd5-4771-a1bf-cbd42599b0fb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:07.646607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:07.702777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 187/378] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:07.704087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff44c9-611c-4973-86f7-075caafdc833 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:07.764374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:07.781308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8d083-2cd5-4771-a1bf-cbd42599b0fb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:07.822340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8fa63-73bd-42e8-a438-842aef701660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:07.823226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8fa63-73bd-42e8-a438-842aef701660 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.823511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8fa63-73bd-42e8-a438-842aef701660 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.823787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8fa63-73bd-42e8-a438-842aef701660 None admin] PATH_INFO: `/v3/projects/fb46b30ed74742b9a95d61944ef39381` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:07.825174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8fa63-73bd-42e8-a438-842aef701660 None admin] RBAC: Authorizing `identity:get_project(project_id=fb46b30ed74742b9a95d61944ef39381)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:07.827389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8fa63-73bd-42e8-a438-842aef701660 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:07.829386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 192/379] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:07 2022] GET /identity/v3/projects/fb46b30ed74742b9a95d61944ef39381 => generated 344 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:07.830299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8fa63-73bd-42e8-a438-842aef701660 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:07.841193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff44c9-611c-4973-86f7-075caafdc833 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:07.882177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbc426-21c8-467e-81f0-365359fb1614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:07.883215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbc426-21c8-467e-81f0-365359fb1614 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.883724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbc426-21c8-467e-81f0-365359fb1614 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.884303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbc426-21c8-467e-81f0-365359fb1614 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:07.885536 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbc426-21c8-467e-81f0-365359fb1614 None admin] RBAC: Adding query filter params (group.id=e0162af404b44cd5b4d8886df243dac7, role.id=c465d586f0ec4b59815b23087d757fd2, scope.project.id=c5de3bc9e442415487283bf9414ae97d) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:07.886075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbc426-21c8-467e-81f0-365359fb1614 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:07.887769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbc426-21c8-467e-81f0-365359fb1614 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:07.894593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 188/380] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Jul 18 16:30:07 2022] GET /identity/v3/role_assignments?group.id=e0162af404b44cd5b4d8886df243dac7&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=c465d586f0ec4b59815b23087d757fd2 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:07.900924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbc426-21c8-467e-81f0-365359fb1614 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:07.955733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8fa63-73bd-42e8-a438-842aef701660 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:07.957504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:07.996677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97771057-0528-40e4-84dd-f361fe1919aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:07.997119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97771057-0528-40e4-84dd-f361fe1919aa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:07.997400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97771057-0528-40e4-84dd-f361fe1919aa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:07.997674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97771057-0528-40e4-84dd-f361fe1919aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:08.025730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97771057-0528-40e4-84dd-f361fe1919aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-97771057-0528-40e4-84dd-f361fe1919aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:08.048908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:08.049873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 193/381] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:08.050737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97771057-0528-40e4-84dd-f361fe1919aa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:08.152647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbc426-21c8-467e-81f0-365359fb1614 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:08.174190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:08.222452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcafd8-fe8c-4645-acab-b0dd6dffa4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:08.223324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcafd8-fe8c-4645-acab-b0dd6dffa4bb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:08.223611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcafd8-fe8c-4645-acab-b0dd6dffa4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:08.223890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcafd8-fe8c-4645-acab-b0dd6dffa4bb None admin] PATH_INFO: `/v3/groups/356b06a90e2e4ebcadc8fde30d2b0dc5` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:08.232722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcafd8-fe8c-4645-acab-b0dd6dffa4bb None admin] RBAC: Authorizing `identity:get_group(group_id=356b06a90e2e4ebcadc8fde30d2b0dc5)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:08.234245 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcafd8-fe8c-4645-acab-b0dd6dffa4bb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:08.237970 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 189/382] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:07 2022] GET /identity/v3/groups/356b06a90e2e4ebcadc8fde30d2b0dc5 => generated 229 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:08.241839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcafd8-fe8c-4645-acab-b0dd6dffa4bb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:08.269737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97771057-0528-40e4-84dd-f361fe1919aa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:08.331212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9053f90-27d2-48ac-8595-4e33da625a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:08.332424 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9053f90-27d2-48ac-8595-4e33da625a6a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:08.332958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9053f90-27d2-48ac-8595-4e33da625a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:08.333461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9053f90-27d2-48ac-8595-4e33da625a6a None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:08.334919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9053f90-27d2-48ac-8595-4e33da625a6a None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:08.335472 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9053f90-27d2-48ac-8595-4e33da625a6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:08.345899 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9053f90-27d2-48ac-8595-4e33da625a6a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:08.366004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 194/383] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:30:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:08.374487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9053f90-27d2-48ac-8595-4e33da625a6a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:08.433340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:08.470133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcafd8-fe8c-4645-acab-b0dd6dffa4bb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:08.530792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9053f90-27d2-48ac-8595-4e33da625a6a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:08.586894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc960f-087e-4321-a6d0-a50e627fde92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:08.587761 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc960f-087e-4321-a6d0-a50e627fde92 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:08.588045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc960f-087e-4321-a6d0-a50e627fde92 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:08.588320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc960f-087e-4321-a6d0-a50e627fde92 None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:08.590879 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc960f-087e-4321-a6d0-a50e627fde92 None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:08.593000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc960f-087e-4321-a6d0-a50e627fde92 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:08.596203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 195/384] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:08 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:08.605213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc648ceb-d23c-483b-b3e3-a67b6c65b10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:08.605879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc648ceb-d23c-483b-b3e3-a67b6c65b10b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:08.606582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc648ceb-d23c-483b-b3e3-a67b6c65b10b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:08.606582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc648ceb-d23c-483b-b3e3-a67b6c65b10b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:08.612903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc648ceb-d23c-483b-b3e3-a67b6c65b10b None admin] RBAC: Adding query filter params (group.id=356b06a90e2e4ebcadc8fde30d2b0dc5, role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=fb46b30ed74742b9a95d61944ef39381) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:08.612903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc648ceb-d23c-483b-b3e3-a67b6c65b10b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:08.612903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc648ceb-d23c-483b-b3e3-a67b6c65b10b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:08.613143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc960f-087e-4321-a6d0-a50e627fde92 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:08.630369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 190/385] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Jul 18 16:30:08 2022] GET /identity/v3/role_assignments?group.id=356b06a90e2e4ebcadc8fde30d2b0dc5&scope.project.id=fb46b30ed74742b9a95d61944ef39381&role.id=81457691f3a2456fa33358cb6a922d05 => generated 260 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:08.802672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:08.811997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc960f-087e-4321-a6d0-a50e627fde92 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:08.884906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c88ca-7928-4908-974b-5aadf1abe410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:08.886129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c88ca-7928-4908-974b-5aadf1abe410 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:08.886610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c88ca-7928-4908-974b-5aadf1abe410 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:08.887105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c88ca-7928-4908-974b-5aadf1abe410 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:08.890337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c88ca-7928-4908-974b-5aadf1abe410 None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:08.892009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c88ca-7928-4908-974b-5aadf1abe410 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:08.893988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 196/386] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:08 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:08.902353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc648ceb-d23c-483b-b3e3-a67b6c65b10b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:08.959066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:09.067451 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc648ceb-d23c-483b-b3e3-a67b6c65b10b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:09.130003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88017d7a-b09c-4764-a76e-c81fdead34a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:09.131133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88017d7a-b09c-4764-a76e-c81fdead34a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:09.131558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88017d7a-b09c-4764-a76e-c81fdead34a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:09.132001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88017d7a-b09c-4764-a76e-c81fdead34a6 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/groups/e0162af404b44cd5b4d8886df243dac7/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:09.138410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88017d7a-b09c-4764-a76e-c81fdead34a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c6a41d535fe491c81d7c8ea25c933bf, group_id=e0162af404b44cd5b4d8886df243dac7, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:09.140090 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88017d7a-b09c-4764-a76e-c81fdead34a6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:09.224762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 191/387] 10.222.0.88 () {58 vars in 1442 bytes} [Mon Jul 18 16:30:08 2022] PUT /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/groups/e0162af404b44cd5b4d8886df243dac7/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:30:09.398862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba0a7-bbeb-43f3-887b-d9563fe44f63 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:09.399356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba0a7-bbeb-43f3-887b-d9563fe44f63 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:09.399356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba0a7-bbeb-43f3-887b-d9563fe44f63 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:09.400871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 197/388] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:09.419023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2da70-bd91-4f59-9080-8747b3761b33 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:09.419346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2da70-bd91-4f59-9080-8747b3761b33 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:09.419600 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2da70-bd91-4f59-9080-8747b3761b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:09.447672 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bbb2da70-bd91-4f59-9080-8747b3761b33 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:09.448911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bbb2da70-bd91-4f59-9080-8747b3761b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:09.545723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 192/389] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:09.577784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ba0a7-bbeb-43f3-887b-d9563fe44f63 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:09.704718 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ba0a7-bbeb-43f3-887b-d9563fe44f63 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:09.739976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81148d-f48b-407a-83ae-2da54fe86be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:09.740797 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81148d-f48b-407a-83ae-2da54fe86be3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:09.741105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81148d-f48b-407a-83ae-2da54fe86be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:09.741383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81148d-f48b-407a-83ae-2da54fe86be3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:09.753261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81148d-f48b-407a-83ae-2da54fe86be3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-0f81148d-f48b-407a-83ae-2da54fe86be3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:09.761502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:09.762370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:09.763150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 198/390] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:30:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:09.769849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2da70-bd91-4f59-9080-8747b3761b33 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:09.813428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:09.911230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2da70-bd91-4f59-9080-8747b3761b33 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:09.971767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44cf5b-59bc-43be-a06b-c42ed43d6558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:09.972377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44cf5b-59bc-43be-a06b-c42ed43d6558 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:09.972659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44cf5b-59bc-43be-a06b-c42ed43d6558 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:09.972959 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44cf5b-59bc-43be-a06b-c42ed43d6558 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:09.973721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44cf5b-59bc-43be-a06b-c42ed43d6558 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:09.974027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44cf5b-59bc-43be-a06b-c42ed43d6558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:09.975414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44cf5b-59bc-43be-a06b-c42ed43d6558 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:09.989437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 193/391] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:09.995354 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81148d-f48b-407a-83ae-2da54fe86be3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:10.037278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:10.139062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f4784-495d-4080-8f45-603ed307abda None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:10.139382 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f4784-495d-4080-8f45-603ed307abda None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:10.139627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f4784-495d-4080-8f45-603ed307abda None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:10.141032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 194/392] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:30:10.145007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81148d-f48b-407a-83ae-2da54fe86be3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:10.146855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63703c8f-a858-4e32-b5e4-c62f1b357a25 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:10.147623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63703c8f-a858-4e32-b5e4-c62f1b357a25 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:10.147922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63703c8f-a858-4e32-b5e4-c62f1b357a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:10.178130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-63703c8f-a858-4e32-b5e4-c62f1b357a25 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:10.179512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-63703c8f-a858-4e32-b5e4-c62f1b357a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:10.183515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80911c9-4962-4f76-8cfd-8e5dcbb50d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:10.186154 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80911c9-4962-4f76-8cfd-8e5dcbb50d07 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:10.186657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80911c9-4962-4f76-8cfd-8e5dcbb50d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:10.187142 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80911c9-4962-4f76-8cfd-8e5dcbb50d07 None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:10.188539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80911c9-4962-4f76-8cfd-8e5dcbb50d07 None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:10.190249 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80911c9-4962-4f76-8cfd-8e5dcbb50d07 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:10.192983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 199/393] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:09 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:10.204904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80911c9-4962-4f76-8cfd-8e5dcbb50d07 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:10.309947 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 195/394] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:10.349745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63703c8f-a858-4e32-b5e4-c62f1b357a25 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:10.383247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:10.389265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:10.417485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80911c9-4962-4f76-8cfd-8e5dcbb50d07 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:10.461661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8175ffc-0e19-4210-beec-e758d86d7339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:10.462540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8175ffc-0e19-4210-beec-e758d86d7339 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:10.462823 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8175ffc-0e19-4210-beec-e758d86d7339 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:10.463102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8175ffc-0e19-4210-beec-e758d86d7339 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:10.464362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8175ffc-0e19-4210-beec-e758d86d7339 None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:10.465910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8175ffc-0e19-4210-beec-e758d86d7339 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:10.468629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 200/395] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:10 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:10.474468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8175ffc-0e19-4210-beec-e758d86d7339 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:10.530416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63703c8f-a858-4e32-b5e4-c62f1b357a25 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:10.568865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4ca7f-8952-49a8-a426-23e9bda561b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:10.570077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4ca7f-8952-49a8-a426-23e9bda561b4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:10.570585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4ca7f-8952-49a8-a426-23e9bda561b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:10.571069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4ca7f-8952-49a8-a426-23e9bda561b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:10.583345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e4ca7f-8952-49a8-a426-23e9bda561b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-92e4ca7f-8952-49a8-a426-23e9bda561b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:10.592717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:10.593650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:10.594416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:10.595025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8175ffc-0e19-4210-beec-e758d86d7339 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:10.597534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 196/396] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:10.599342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4a6f3-a84b-44a2-97ee-f6930a7ca33c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:10.599863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4a6f3-a84b-44a2-97ee-f6930a7ca33c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:10.600340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4a6f3-a84b-44a2-97ee-f6930a7ca33c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:10.602059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 197/397] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:10.608740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce003c5-2f33-4992-8a26-ad936fb28c58 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:10.609410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce003c5-2f33-4992-8a26-ad936fb28c58 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:10.609898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce003c5-2f33-4992-8a26-ad936fb28c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:10.628977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2ce003c5-2f33-4992-8a26-ad936fb28c58 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:10.630756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2ce003c5-2f33-4992-8a26-ad936fb28c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:10.638590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e93ba-523e-4d77-8373-9cb2db8e9640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:10.639583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e93ba-523e-4d77-8373-9cb2db8e9640 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:10.639875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e93ba-523e-4d77-8373-9cb2db8e9640 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:10.640154 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e93ba-523e-4d77-8373-9cb2db8e9640 None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/groups/e0162af404b44cd5b4d8886df243dac7/roles/c465d586f0ec4b59815b23087d757fd2` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:10.642814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e93ba-523e-4d77-8373-9cb2db8e9640 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c6a41d535fe491c81d7c8ea25c933bf, group_id=e0162af404b44cd5b4d8886df243dac7, role_id=c465d586f0ec4b59815b23087d757fd2)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:10.644426 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e93ba-523e-4d77-8373-9cb2db8e9640 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:10.712909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 201/398] 10.222.0.88 () {58 vars in 1442 bytes} [Mon Jul 18 16:30:10 2022] PUT /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf/groups/e0162af404b44cd5b4d8886df243dac7/roles/c465d586f0ec4b59815b23087d757fd2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:30:10.712909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e93ba-523e-4d77-8373-9cb2db8e9640 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:10.728247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 198/399] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:10.747200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:10.759869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce003c5-2f33-4992-8a26-ad936fb28c58 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:10.779142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:10.990530 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e93ba-523e-4d77-8373-9cb2db8e9640 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:11.032600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62c1c3-7112-4a86-95be-546808a45db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:11.036916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62c1c3-7112-4a86-95be-546808a45db8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.036916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62c1c3-7112-4a86-95be-546808a45db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.036916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62c1c3-7112-4a86-95be-546808a45db8 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.036916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62c1c3-7112-4a86-95be-546808a45db8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:11.036916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62c1c3-7112-4a86-95be-546808a45db8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:11.036916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62c1c3-7112-4a86-95be-546808a45db8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:11.041789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce003c5-2f33-4992-8a26-ad936fb28c58 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:11.060861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 202/400] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:30:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:11.066923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62c1c3-7112-4a86-95be-546808a45db8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:11.084971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc2b97-99c8-4a6a-afcb-3933a472ec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:11.085798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc2b97-99c8-4a6a-afcb-3933a472ec3f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.086081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc2b97-99c8-4a6a-afcb-3933a472ec3f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.086358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc2b97-99c8-4a6a-afcb-3933a472ec3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.099247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc2b97-99c8-4a6a-afcb-3933a472ec3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c2fc2b97-99c8-4a6a-afcb-3933a472ec3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:11.108183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:11.109120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:11.110361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 199/401] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:30:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:11.115398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc2b97-99c8-4a6a-afcb-3933a472ec3f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:11.215472 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:11.243919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62c1c3-7112-4a86-95be-546808a45db8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:11.287076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:11.290747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8438e8-4ddd-4d15-b315-1eb1dfebd29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:11.291640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8438e8-4ddd-4d15-b315-1eb1dfebd29c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.291925 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8438e8-4ddd-4d15-b315-1eb1dfebd29c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.292203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8438e8-4ddd-4d15-b315-1eb1dfebd29c None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.296912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8438e8-4ddd-4d15-b315-1eb1dfebd29c None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:11.306275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8438e8-4ddd-4d15-b315-1eb1dfebd29c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:11.306275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 203/402] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:11 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:11.306275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8438e8-4ddd-4d15-b315-1eb1dfebd29c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:11.323869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc2b97-99c8-4a6a-afcb-3933a472ec3f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:11.365636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ff44e-01fe-4d47-bb02-3ad4b866f9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:11.366717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ff44e-01fe-4d47-bb02-3ad4b866f9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.367378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ff44e-01fe-4d47-bb02-3ad4b866f9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.368962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ff44e-01fe-4d47-bb02-3ad4b866f9b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.369923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ff44e-01fe-4d47-bb02-3ad4b866f9b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:11.370825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ff44e-01fe-4d47-bb02-3ad4b866f9b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:11.373237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ff44e-01fe-4d47-bb02-3ad4b866f9b4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:11.394221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 200/403] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:11.403291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ff44e-01fe-4d47-bb02-3ad4b866f9b4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:11.474759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8438e8-4ddd-4d15-b315-1eb1dfebd29c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:11.531147 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ff44e-01fe-4d47-bb02-3ad4b866f9b4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:11.541736 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1301d90-3fa1-4127-aa3d-5359e5567740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:11.542643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1301d90-3fa1-4127-aa3d-5359e5567740 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.542991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1301d90-3fa1-4127-aa3d-5359e5567740 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.543275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1301d90-3fa1-4127-aa3d-5359e5567740 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.544499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1301d90-3fa1-4127-aa3d-5359e5567740 None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:11.546964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1301d90-3fa1-4127-aa3d-5359e5567740 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:11.548773 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 204/404] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:11 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:11.550135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6994ee-a2eb-4704-a710-b73c45e49b24 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.550426 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6994ee-a2eb-4704-a710-b73c45e49b24 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.550708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6994ee-a2eb-4704-a710-b73c45e49b24 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.552092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 205/405] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:11.557411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4581061-b017-4475-9649-2300b08d5a97 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.557703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4581061-b017-4475-9649-2300b08d5a97 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.557991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4581061-b017-4475-9649-2300b08d5a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.568795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:11.575521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa6ce1-d763-4d6f-9af7-5847b8ca98e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:11.577259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa6ce1-d763-4d6f-9af7-5847b8ca98e7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.577634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa6ce1-d763-4d6f-9af7-5847b8ca98e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.577977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa6ce1-d763-4d6f-9af7-5847b8ca98e7 None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.579455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa6ce1-d763-4d6f-9af7-5847b8ca98e7 None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:11.582608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa6ce1-d763-4d6f-9af7-5847b8ca98e7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:11.582854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 201/406] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:11 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:11.583686 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa6ce1-d763-4d6f-9af7-5847b8ca98e7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:11.585948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c4581061-b017-4475-9649-2300b08d5a97 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:11.587734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c4581061-b017-4475-9649-2300b08d5a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:11.598866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:11.660933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 206/407] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:30:11.661775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4581061-b017-4475-9649-2300b08d5a97 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:11.699300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa6ce1-d763-4d6f-9af7-5847b8ca98e7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:11.735302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6d8a6-b537-4651-a005-9acc95f3af17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:11.736104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6d8a6-b537-4651-a005-9acc95f3af17 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.736394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6d8a6-b537-4651-a005-9acc95f3af17 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.736673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6d8a6-b537-4651-a005-9acc95f3af17 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d/groups/e0162af404b44cd5b4d8886df243dac7/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.738686 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6d8a6-b537-4651-a005-9acc95f3af17 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5de3bc9e442415487283bf9414ae97d, group_id=e0162af404b44cd5b4d8886df243dac7, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:11.740123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6d8a6-b537-4651-a005-9acc95f3af17 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:11.774184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4581061-b017-4475-9649-2300b08d5a97 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:11.809472 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f783e-5ad6-4acb-b268-8201e65e6c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:11.810383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f783e-5ad6-4acb-b268-8201e65e6c04 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.810668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f783e-5ad6-4acb-b268-8201e65e6c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.810943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f783e-5ad6-4acb-b268-8201e65e6c04 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.812664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f783e-5ad6-4acb-b268-8201e65e6c04 None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:11.814150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f783e-5ad6-4acb-b268-8201e65e6c04 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:11.815845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 207/408] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:11 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:11.817032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b031c-9fdc-45f3-b973-7fe80701bd16 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:11.817321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b031c-9fdc-45f3-b973-7fe80701bd16 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:11.817598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b031c-9fdc-45f3-b973-7fe80701bd16 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:11.823048 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 208/409] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:11.823048 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b031c-9fdc-45f3-b973-7fe80701bd16 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:11.835967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:11.837272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 202/410] 10.222.0.88 () {58 vars in 1442 bytes} [Mon Jul 18 16:30:11 2022] PUT /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d/groups/e0162af404b44cd5b4d8886df243dac7/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:30:11.838272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6d8a6-b537-4651-a005-9acc95f3af17 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:11.915035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:11.971198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6d8a6-b537-4651-a005-9acc95f3af17 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:11.989961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b031c-9fdc-45f3-b973-7fe80701bd16 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:12.020956 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b7f50-1596-4337-bc45-cbe62ed09acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:12.021921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7f50-1596-4337-bc45-cbe62ed09acd None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:12.022215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7f50-1596-4337-bc45-cbe62ed09acd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:12.022497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b7f50-1596-4337-bc45-cbe62ed09acd None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d/groups/e0162af404b44cd5b4d8886df243dac7/roles/c465d586f0ec4b59815b23087d757fd2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:12.024851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b7f50-1596-4337-bc45-cbe62ed09acd None admin] RBAC: Authorizing `identity:create_grant(project_id=c5de3bc9e442415487283bf9414ae97d, group_id=e0162af404b44cd5b4d8886df243dac7, role_id=c465d586f0ec4b59815b23087d757fd2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:12.026330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b7f50-1596-4337-bc45-cbe62ed09acd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:12.037456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff76b9d-9e39-4bf8-acea-37bdbbdae609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:12.038256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76b9d-9e39-4bf8-acea-37bdbbdae609 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:12.038540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76b9d-9e39-4bf8-acea-37bdbbdae609 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:12.038816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff76b9d-9e39-4bf8-acea-37bdbbdae609 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:12.050827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff76b9d-9e39-4bf8-acea-37bdbbdae609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-fff76b9d-9e39-4bf8-acea-37bdbbdae609 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:12.058733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:12.059595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:12.060497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 209/411] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:12.065038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62794665-b511-4309-bceb-16643d434863 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:12.065038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62794665-b511-4309-bceb-16643d434863 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:12.065038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62794665-b511-4309-bceb-16643d434863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:12.082145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-62794665-b511-4309-bceb-16643d434863 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:12.083357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-62794665-b511-4309-bceb-16643d434863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:12.102800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 203/412] 10.222.0.88 () {58 vars in 1442 bytes} [Mon Jul 18 16:30:11 2022] PUT /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d/groups/e0162af404b44cd5b4d8886df243dac7/roles/c465d586f0ec4b59815b23087d757fd2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:30:12.103983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b7f50-1596-4337-bc45-cbe62ed09acd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:12.214959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 210/413] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:12.238752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62794665-b511-4309-bceb-16643d434863 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:12.241564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:12.277398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b7f50-1596-4337-bc45-cbe62ed09acd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:12.307607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:12.334382 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7667c-ef21-4e06-b91a-4ef18363561a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:12.335263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7667c-ef21-4e06-b91a-4ef18363561a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:12.335647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7667c-ef21-4e06-b91a-4ef18363561a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:12.336029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7667c-ef21-4e06-b91a-4ef18363561a None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:12.336937 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7667c-ef21-4e06-b91a-4ef18363561a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:12.337383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7667c-ef21-4e06-b91a-4ef18363561a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:12.339116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae7667c-ef21-4e06-b91a-4ef18363561a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:12.360762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 204/414] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:12.368317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7667c-ef21-4e06-b91a-4ef18363561a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:12.372332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62794665-b511-4309-bceb-16643d434863 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:12.415669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8097839-dea4-4cd6-946a-c795f1445b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:12.416868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8097839-dea4-4cd6-946a-c795f1445b88 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:12.417382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8097839-dea4-4cd6-946a-c795f1445b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:12.417868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8097839-dea4-4cd6-946a-c795f1445b88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:12.429450 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8097839-dea4-4cd6-946a-c795f1445b88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c8097839-dea4-4cd6-946a-c795f1445b88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:12.438218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:12.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:12.439905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:12.442713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 211/415] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:12.448635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8097839-dea4-4cd6-946a-c795f1445b88 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:12.494692 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7667c-ef21-4e06-b91a-4ef18363561a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:12.517746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:12.540360 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae648ced-b111-4e42-ac0d-6148a42ff030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:12.541343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae648ced-b111-4e42-ac0d-6148a42ff030 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:12.541635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae648ced-b111-4e42-ac0d-6148a42ff030 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:12.541921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae648ced-b111-4e42-ac0d-6148a42ff030 None admin] PATH_INFO: `/v3/groups/356b06a90e2e4ebcadc8fde30d2b0dc5` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:12.543144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae648ced-b111-4e42-ac0d-6148a42ff030 None admin] RBAC: Authorizing `identity:get_group(group_id=356b06a90e2e4ebcadc8fde30d2b0dc5)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:12.544728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae648ced-b111-4e42-ac0d-6148a42ff030 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:12.548040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 205/416] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:12 2022] GET /identity/v3/groups/356b06a90e2e4ebcadc8fde30d2b0dc5 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:12.556423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae648ced-b111-4e42-ac0d-6148a42ff030 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:12.568597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8097839-dea4-4cd6-946a-c795f1445b88 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:12.607452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465bbf7-d378-4a5f-95a9-ef771a0cac05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:12.608498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465bbf7-d378-4a5f-95a9-ef771a0cac05 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:12.609042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465bbf7-d378-4a5f-95a9-ef771a0cac05 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:12.609522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465bbf7-d378-4a5f-95a9-ef771a0cac05 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:12.610511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465bbf7-d378-4a5f-95a9-ef771a0cac05 None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:12.611020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465bbf7-d378-4a5f-95a9-ef771a0cac05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:12.612631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465bbf7-d378-4a5f-95a9-ef771a0cac05 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:12.622089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 212/417] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:30:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:12.628536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465bbf7-d378-4a5f-95a9-ef771a0cac05 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:12.657722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:12.734351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae648ced-b111-4e42-ac0d-6148a42ff030 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:12.780857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465bbf7-d378-4a5f-95a9-ef771a0cac05 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:12.784459 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d784da5-904f-45f5-9dec-a4d54452d214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:12.785676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d784da5-904f-45f5-9dec-a4d54452d214 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:12.786482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d784da5-904f-45f5-9dec-a4d54452d214 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:12.788343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d784da5-904f-45f5-9dec-a4d54452d214 None admin] PATH_INFO: `/v3/projects/fb46b30ed74742b9a95d61944ef39381` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:12.795580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d784da5-904f-45f5-9dec-a4d54452d214 None admin] RBAC: Authorizing `identity:get_project(project_id=fb46b30ed74742b9a95d61944ef39381)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:12.797392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d784da5-904f-45f5-9dec-a4d54452d214 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:12.800712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 206/418] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:12 2022] GET /identity/v3/projects/fb46b30ed74742b9a95d61944ef39381 => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:12.808960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d784da5-904f-45f5-9dec-a4d54452d214 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:12.840447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:12.873617 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd41b5e-dbd5-4fe4-b13f-883e12a3d8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:12.874287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd41b5e-dbd5-4fe4-b13f-883e12a3d8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:12.874577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd41b5e-dbd5-4fe4-b13f-883e12a3d8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:12.874866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd41b5e-dbd5-4fe4-b13f-883e12a3d8c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:12.950736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d784da5-904f-45f5-9dec-a4d54452d214 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:12.979819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:13.003702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a4bcd-f5ef-4275-ac2d-8d5f9b6313e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:13.004400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a4bcd-f5ef-4275-ac2d-8d5f9b6313e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:13.004689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a4bcd-f5ef-4275-ac2d-8d5f9b6313e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:13.004995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a4bcd-f5ef-4275-ac2d-8d5f9b6313e0 None admin] PATH_INFO: `/v3/projects/fb46b30ed74742b9a95d61944ef39381/groups/356b06a90e2e4ebcadc8fde30d2b0dc5/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:13.007481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a4bcd-f5ef-4275-ac2d-8d5f9b6313e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=fb46b30ed74742b9a95d61944ef39381, group_id=356b06a90e2e4ebcadc8fde30d2b0dc5, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:13.009319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a4bcd-f5ef-4275-ac2d-8d5f9b6313e0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:13.022937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd41b5e-dbd5-4fe4-b13f-883e12a3d8c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:13.024633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd41b5e-dbd5-4fe4-b13f-883e12a3d8c6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:13.095790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 207/419] 10.222.0.88 () {58 vars in 1442 bytes} [Mon Jul 18 16:30:12 2022] PUT /identity/v3/projects/fb46b30ed74742b9a95d61944ef39381/groups/356b06a90e2e4ebcadc8fde30d2b0dc5/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:30:13.098824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be6fe3-11fe-4ce6-a3c7-11e3e7652364 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:13.099958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be6fe3-11fe-4ce6-a3c7-11e3e7652364 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:13.100579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be6fe3-11fe-4ce6-a3c7-11e3e7652364 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:13.104288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 208/420] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:13.111725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f833ff5-ee92-47fb-9df5-05028b9d6ae5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:13.112045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f833ff5-ee92-47fb-9df5-05028b9d6ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:13.112302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f833ff5-ee92-47fb-9df5-05028b9d6ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:13.159514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 213/421] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:30:13.165873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd41b5e-dbd5-4fe4-b13f-883e12a3d8c6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:13.167578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0f833ff5-ee92-47fb-9df5-05028b9d6ae5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:13.168986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0f833ff5-ee92-47fb-9df5-05028b9d6ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:13.197272 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:13.287491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 209/422] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:13.301933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd41b5e-dbd5-4fe4-b13f-883e12a3d8c6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:13.318961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f833ff5-ee92-47fb-9df5-05028b9d6ae5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:13.340411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e953fe8-4255-4c0a-89e4-e93296d6cbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:13.341305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e953fe8-4255-4c0a-89e4-e93296d6cbab None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:13.342119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e953fe8-4255-4c0a-89e4-e93296d6cbab None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:13.342394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e953fe8-4255-4c0a-89e4-e93296d6cbab None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:13.343659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e953fe8-4255-4c0a-89e4-e93296d6cbab None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:13.345267 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e953fe8-4255-4c0a-89e4-e93296d6cbab None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:13.347177 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 214/423] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:13 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:13.352589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e953fe8-4255-4c0a-89e4-e93296d6cbab None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:13.505218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f833ff5-ee92-47fb-9df5-05028b9d6ae5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:13.516400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:13.543288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:13.550384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd9a62-a339-4fa1-8bd5-13180ee04ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:13.552367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd9a62-a339-4fa1-8bd5-13180ee04ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:13.552367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd9a62-a339-4fa1-8bd5-13180ee04ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:13.552367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd9a62-a339-4fa1-8bd5-13180ee04ac7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:13.568198 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e953fe8-4255-4c0a-89e4-e93296d6cbab None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:13.581821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd9a62-a339-4fa1-8bd5-13180ee04ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-decd9a62-a339-4fa1-8bd5-13180ee04ac7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:13.591193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:13.592177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:13.593196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:13.598723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 210/424] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:30:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:13.607005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd9a62-a339-4fa1-8bd5-13180ee04ac7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:13.641726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844075d3-fb1a-4753-8fcc-ed7caa1200d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:13.642647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844075d3-fb1a-4753-8fcc-ed7caa1200d9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:13.642928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844075d3-fb1a-4753-8fcc-ed7caa1200d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:13.643199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844075d3-fb1a-4753-8fcc-ed7caa1200d9 None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:13.644560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844075d3-fb1a-4753-8fcc-ed7caa1200d9 None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:13.647353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844075d3-fb1a-4753-8fcc-ed7caa1200d9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:13.657941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 215/425] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:13 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:13.658890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844075d3-fb1a-4753-8fcc-ed7caa1200d9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:13.793766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd9a62-a339-4fa1-8bd5-13180ee04ac7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:13.859805 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844075d3-fb1a-4753-8fcc-ed7caa1200d9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:13.874901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c180892-7f27-42c9-a34b-1eaa217b3729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:13.875587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c180892-7f27-42c9-a34b-1eaa217b3729 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:13.875872 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c180892-7f27-42c9-a34b-1eaa217b3729 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:13.876204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c180892-7f27-42c9-a34b-1eaa217b3729 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:13.877064 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c180892-7f27-42c9-a34b-1eaa217b3729 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:13.877476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c180892-7f27-42c9-a34b-1eaa217b3729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:13.878980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c180892-7f27-42c9-a34b-1eaa217b3729 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:13.900518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 211/426] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:13.906890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c180892-7f27-42c9-a34b-1eaa217b3729 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:13.912058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8a48c-e40e-4e15-a5a4-bc4ad6057a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:13.912955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8a48c-e40e-4e15-a5a4-bc4ad6057a60 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:13.913384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8a48c-e40e-4e15-a5a4-bc4ad6057a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:13.913843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8a48c-e40e-4e15-a5a4-bc4ad6057a60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:13.914834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8a48c-e40e-4e15-a5a4-bc4ad6057a60 None admin] RBAC: Adding query filter params (group.id=e0162af404b44cd5b4d8886df243dac7, role.id=3b2e1fad42f441e9a61c142477a50b11, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:13.915397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8a48c-e40e-4e15-a5a4-bc4ad6057a60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:13.917023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8a48c-e40e-4e15-a5a4-bc4ad6057a60 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:13.935979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 216/427] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Jul 18 16:30:13 2022] GET /identity/v3/role_assignments?group.id=e0162af404b44cd5b4d8886df243dac7&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 611 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:13.948494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:14.004125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd0189-541f-49da-8779-736acd064b3f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.004707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd0189-541f-49da-8779-736acd064b3f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.005014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd0189-541f-49da-8779-736acd064b3f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.006534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 217/428] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:30:14.011740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69322ab0-e93a-4407-aea5-a790c0aff7ef None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.012030 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69322ab0-e93a-4407-aea5-a790c0aff7ef None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.012313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69322ab0-e93a-4407-aea5-a790c0aff7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.028190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c180892-7f27-42c9-a34b-1eaa217b3729 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:14.053940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-69322ab0-e93a-4407-aea5-a790c0aff7ef None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:14.055193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-69322ab0-e93a-4407-aea5-a790c0aff7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:14.075491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ccb0a-af3a-4ac0-82cc-aa0788c84b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:14.076778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ccb0a-af3a-4ac0-82cc-aa0788c84b66 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.077089 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ccb0a-af3a-4ac0-82cc-aa0788c84b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.077370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ccb0a-af3a-4ac0-82cc-aa0788c84b66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.097311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:14.163848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ccb0a-af3a-4ac0-82cc-aa0788c84b66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:14.165315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ccb0a-af3a-4ac0-82cc-aa0788c84b66 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:14.179818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 218/429] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:30:14.205985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:14.207101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69322ab0-e93a-4407-aea5-a790c0aff7ef None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:14.299117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 212/430] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:14.306842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ccb0a-af3a-4ac0-82cc-aa0788c84b66 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:14.391722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69322ab0-e93a-4407-aea5-a790c0aff7ef None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:14.436803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766a33d-d578-4351-8e3f-d7d3f2fe7942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:14.437573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766a33d-d578-4351-8e3f-d7d3f2fe7942 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.437859 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766a33d-d578-4351-8e3f-d7d3f2fe7942 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.438139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766a33d-d578-4351-8e3f-d7d3f2fe7942 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.452179 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766a33d-d578-4351-8e3f-d7d3f2fe7942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:14.453734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ccb0a-af3a-4ac0-82cc-aa0788c84b66 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9766a33d-d578-4351-8e3f-d7d3f2fe7942 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:14.460702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:14.461638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:14.462420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 219/431] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:14.470009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766a33d-d578-4351-8e3f-d7d3f2fe7942 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:14.491431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:14.502307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:14.511881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee029f-4230-4897-af63-356d834dc7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:14.514494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee029f-4230-4897-af63-356d834dc7bf None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.514977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee029f-4230-4897-af63-356d834dc7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.515406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee029f-4230-4897-af63-356d834dc7bf None admin] PATH_INFO: `/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.517203 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee029f-4230-4897-af63-356d834dc7bf None admin] RBAC: Authorizing `identity:get_project(project_id=1c6a41d535fe491c81d7c8ea25c933bf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:14.522564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee029f-4230-4897-af63-356d834dc7bf None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:14.526552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 213/432] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:14 2022] GET /identity/v3/projects/1c6a41d535fe491c81d7c8ea25c933bf => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:14.536653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee029f-4230-4897-af63-356d834dc7bf None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:14.668907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766a33d-d578-4351-8e3f-d7d3f2fe7942 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:14.702593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee029f-4230-4897-af63-356d834dc7bf None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:14.712907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046ca0c-ebb4-4e6a-9b59-4c9ab13c62ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:14.712907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046ca0c-ebb4-4e6a-9b59-4c9ab13c62ed None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.712907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046ca0c-ebb4-4e6a-9b59-4c9ab13c62ed None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.712907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046ca0c-ebb4-4e6a-9b59-4c9ab13c62ed None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.713256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046ca0c-ebb4-4e6a-9b59-4c9ab13c62ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:14.713256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046ca0c-ebb4-4e6a-9b59-4c9ab13c62ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:14.713256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046ca0c-ebb4-4e6a-9b59-4c9ab13c62ed None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:14.723682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 220/433] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:30:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:14.728959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046ca0c-ebb4-4e6a-9b59-4c9ab13c62ed None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:14.751921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd0040-c340-4fbc-b7e3-fa7db6f5d65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:14.753379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd0040-c340-4fbc-b7e3-fa7db6f5d65d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.754851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd0040-c340-4fbc-b7e3-fa7db6f5d65d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.754851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd0040-c340-4fbc-b7e3-fa7db6f5d65d None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.755330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd0040-c340-4fbc-b7e3-fa7db6f5d65d None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:14.756802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd0040-c340-4fbc-b7e3-fa7db6f5d65d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:14.758889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 214/434] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:14 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:14.765418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd0040-c340-4fbc-b7e3-fa7db6f5d65d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:14.838793 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:14.856063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:14.870002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046ca0c-ebb4-4e6a-9b59-4c9ab13c62ed None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:14.898228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd0040-c340-4fbc-b7e3-fa7db6f5d65d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:14.910245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf7f37-65f6-43fb-b45f-c33f775c8a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:14.911258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf7f37-65f6-43fb-b45f-c33f775c8a5e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.911368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf7f37-65f6-43fb-b45f-c33f775c8a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.911716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf7f37-65f6-43fb-b45f-c33f775c8a5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.944704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b8c89-304d-48a2-a822-f5650666fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:14.945509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b8c89-304d-48a2-a822-f5650666fde4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.945868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b8c89-304d-48a2-a822-f5650666fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.946218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b8c89-304d-48a2-a822-f5650666fde4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.947124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b8c89-304d-48a2-a822-f5650666fde4 None admin] RBAC: Adding query filter params (group.id=e0162af404b44cd5b4d8886df243dac7, role.id=c465d586f0ec4b59815b23087d757fd2, scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:14.947440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b8c89-304d-48a2-a822-f5650666fde4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:14.948905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b8c89-304d-48a2-a822-f5650666fde4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:14.955481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 215/435] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Jul 18 16:30:14 2022] GET /identity/v3/role_assignments?group.id=e0162af404b44cd5b4d8886df243dac7&scope.project.id=1c6a41d535fe491c81d7c8ea25c933bf&role.id=c465d586f0ec4b59815b23087d757fd2 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:14.956663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c8385-c50d-4c6f-8103-53cff51f23f0 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.956981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c8385-c50d-4c6f-8103-53cff51f23f0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.957262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c8385-c50d-4c6f-8103-53cff51f23f0 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.961036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 216/436] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:14.971267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ea60e-b6f8-4632-9ff3-9d27c28e1be5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:14.971584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ea60e-b6f8-4632-9ff3-9d27c28e1be5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:14.971827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ea60e-b6f8-4632-9ff3-9d27c28e1be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:14.995853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-276ea60e-b6f8-4632-9ff3-9d27c28e1be5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:14.997846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-276ea60e-b6f8-4632-9ff3-9d27c28e1be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:14.999753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf7f37-65f6-43fb-b45f-c33f775c8a5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:15.001848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf7f37-65f6-43fb-b45f-c33f775c8a5e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:15.134176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 217/437] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:30:15.152074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ea60e-b6f8-4632-9ff3-9d27c28e1be5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:15.169412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:15.209380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 221/438] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:30:15.220939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf7f37-65f6-43fb-b45f-c33f775c8a5e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:15.255401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:15.323112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ea60e-b6f8-4632-9ff3-9d27c28e1be5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:15.395234 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5454aa2-9d34-48d1-b152-2e98af4be641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:15.395980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf7f37-65f6-43fb-b45f-c33f775c8a5e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:15.396089 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5454aa2-9d34-48d1-b152-2e98af4be641 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:15.396338 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5454aa2-9d34-48d1-b152-2e98af4be641 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:15.396612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5454aa2-9d34-48d1-b152-2e98af4be641 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:15.416909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5454aa2-9d34-48d1-b152-2e98af4be641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b5454aa2-9d34-48d1-b152-2e98af4be641 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:15.422260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:15.423169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:15.423983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:15.427871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 218/439] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:30:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:15.434936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5454aa2-9d34-48d1-b152-2e98af4be641 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:15.449444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5665a6-265b-483a-a9d0-9874e2214c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:15.450306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5665a6-265b-483a-a9d0-9874e2214c11 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:15.450661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5665a6-265b-483a-a9d0-9874e2214c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:15.451626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5665a6-265b-483a-a9d0-9874e2214c11 None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:15.452991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5665a6-265b-483a-a9d0-9874e2214c11 None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:15.454553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5665a6-265b-483a-a9d0-9874e2214c11 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:15.458712 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 222/440] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:15 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:15.464191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5665a6-265b-483a-a9d0-9874e2214c11 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:15.514553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:15.551682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5454aa2-9d34-48d1-b152-2e98af4be641 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:15.579558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:15.580659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5665a6-265b-483a-a9d0-9874e2214c11 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:15.594499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d807c-e065-419d-b9be-1c2c6e7e8b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:15.595175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d807c-e065-419d-b9be-1c2c6e7e8b9d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:15.595460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d807c-e065-419d-b9be-1c2c6e7e8b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:15.595853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d807c-e065-419d-b9be-1c2c6e7e8b9d None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:15.598844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d807c-e065-419d-b9be-1c2c6e7e8b9d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:15.598844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d807c-e065-419d-b9be-1c2c6e7e8b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:15.598844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d807c-e065-419d-b9be-1c2c6e7e8b9d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:15.608658 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 219/441] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:15.614120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d807c-e065-419d-b9be-1c2c6e7e8b9d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:15.618258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df08a3-75cf-4b07-897e-d5625bee0d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:15.619129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df08a3-75cf-4b07-897e-d5625bee0d26 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:15.619412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df08a3-75cf-4b07-897e-d5625bee0d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:15.619689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df08a3-75cf-4b07-897e-d5625bee0d26 None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:15.620913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df08a3-75cf-4b07-897e-d5625bee0d26 None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:15.622425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df08a3-75cf-4b07-897e-d5625bee0d26 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:15.624338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 223/442] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:15 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:15.625601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4d30-1599-49fd-8b0c-d28e0e2cc7b7 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:15.625891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4d30-1599-49fd-8b0c-d28e0e2cc7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:15.626169 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b4d30-1599-49fd-8b0c-d28e0e2cc7b7 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:15.627416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 224/443] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:15.633038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b4d30-1599-49fd-8b0c-d28e0e2cc7b7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:15.732977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d807c-e065-419d-b9be-1c2c6e7e8b9d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:15.736910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b4d30-1599-49fd-8b0c-d28e0e2cc7b7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:15.750489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:15.769010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708e25c-9edb-448a-a91b-b4a11ec6d0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:15.769664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708e25c-9edb-448a-a91b-b4a11ec6d0bd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:15.769948 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708e25c-9edb-448a-a91b-b4a11ec6d0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:15.770229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708e25c-9edb-448a-a91b-b4a11ec6d0bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:15.771460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c06f7-1086-440f-a604-aefb75955ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:15.772145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c06f7-1086-440f-a604-aefb75955ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:15.772476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c06f7-1086-440f-a604-aefb75955ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:15.772764 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c06f7-1086-440f-a604-aefb75955ce7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:15.773649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c06f7-1086-440f-a604-aefb75955ce7 None admin] RBAC: Adding query filter params (group.id=e0162af404b44cd5b4d8886df243dac7, role.id=3b2e1fad42f441e9a61c142477a50b11, scope.project.id=c5de3bc9e442415487283bf9414ae97d) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:15.773962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c06f7-1086-440f-a604-aefb75955ce7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:15.775414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c06f7-1086-440f-a604-aefb75955ce7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:15.783613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 225/444] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Jul 18 16:30:15 2022] GET /identity/v3/role_assignments?group.id=e0162af404b44cd5b4d8886df243dac7&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=3b2e1fad42f441e9a61c142477a50b11 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:15.785243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00e87e-ce0b-4b02-8206-3418e006751f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:15.785923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00e87e-ce0b-4b02-8206-3418e006751f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:15.786677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00e87e-ce0b-4b02-8206-3418e006751f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:15.805313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ff00e87e-ce0b-4b02-8206-3418e006751f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:15.806727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ff00e87e-ce0b-4b02-8206-3418e006751f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:15.830766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:15.852186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708e25c-9edb-448a-a91b-b4a11ec6d0bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:15.853811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708e25c-9edb-448a-a91b-b4a11ec6d0bd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:15.896946 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 226/445] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:15.913200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00e87e-ce0b-4b02-8206-3418e006751f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:15.997507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 220/446] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:30:16.003769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708e25c-9edb-448a-a91b-b4a11ec6d0bd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:16.046002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00e87e-ce0b-4b02-8206-3418e006751f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:16.055113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:16.091269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932bb46-4d50-4854-ab2d-d184b45d36dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:16.093177 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932bb46-4d50-4854-ab2d-d184b45d36dd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:16.093177 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932bb46-4d50-4854-ab2d-d184b45d36dd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:16.093343 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932bb46-4d50-4854-ab2d-d184b45d36dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:16.106586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932bb46-4d50-4854-ab2d-d184b45d36dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:16.108456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a932bb46-4d50-4854-ab2d-d184b45d36dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:16.115671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:16.119303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:16.120152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 227/447] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:16.124838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932bb46-4d50-4854-ab2d-d184b45d36dd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:16.135083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708e25c-9edb-448a-a91b-b4a11ec6d0bd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:16.179398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3af55-7fc0-47e5-a720-e7fa034014af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:16.180342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3af55-7fc0-47e5-a720-e7fa034014af None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:16.180628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3af55-7fc0-47e5-a720-e7fa034014af None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:16.181021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3af55-7fc0-47e5-a720-e7fa034014af None admin] PATH_INFO: `/v3/projects/c5de3bc9e442415487283bf9414ae97d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:16.182240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3af55-7fc0-47e5-a720-e7fa034014af None admin] RBAC: Authorizing `identity:get_project(project_id=c5de3bc9e442415487283bf9414ae97d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:16.183677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3af55-7fc0-47e5-a720-e7fa034014af None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:16.185995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 221/448] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:16 2022] GET /identity/v3/projects/c5de3bc9e442415487283bf9414ae97d => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:16.191496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3af55-7fc0-47e5-a720-e7fa034014af None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:16.232614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932bb46-4d50-4854-ab2d-d184b45d36dd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:16.268055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dc124-8add-49eb-9bdb-2b38acd45e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:16.269251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dc124-8add-49eb-9bdb-2b38acd45e34 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:16.269541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dc124-8add-49eb-9bdb-2b38acd45e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:16.269823 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dc124-8add-49eb-9bdb-2b38acd45e34 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:16.270585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dc124-8add-49eb-9bdb-2b38acd45e34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:16.270894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dc124-8add-49eb-9bdb-2b38acd45e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:16.272305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dc124-8add-49eb-9bdb-2b38acd45e34 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:16.281767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 228/449] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:16.287464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dc124-8add-49eb-9bdb-2b38acd45e34 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:16.304545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3af55-7fc0-47e5-a720-e7fa034014af None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:16.316701 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:16.339639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf7aff-2d9f-4c7a-9980-34975062e35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:16.340510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf7aff-2d9f-4c7a-9980-34975062e35e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:16.340799 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf7aff-2d9f-4c7a-9980-34975062e35e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:16.341118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf7aff-2d9f-4c7a-9980-34975062e35e None admin] PATH_INFO: `/v3/groups/e0162af404b44cd5b4d8886df243dac7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:16.342894 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf7aff-2d9f-4c7a-9980-34975062e35e None admin] RBAC: Authorizing `identity:get_group(group_id=e0162af404b44cd5b4d8886df243dac7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:16.344346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf7aff-2d9f-4c7a-9980-34975062e35e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:16.346026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 222/450] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:16 2022] GET /identity/v3/groups/e0162af404b44cd5b4d8886df243dac7 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:16.351795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf7aff-2d9f-4c7a-9980-34975062e35e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:16.371298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:16.391518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dc124-8add-49eb-9bdb-2b38acd45e34 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:16.427037 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe4d83-1216-4946-bbe3-a63d4c3d460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:16.427740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe4d83-1216-4946-bbe3-a63d4c3d460c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:16.427962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe4d83-1216-4946-bbe3-a63d4c3d460c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:16.428239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe4d83-1216-4946-bbe3-a63d4c3d460c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:16.454173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf7aff-2d9f-4c7a-9980-34975062e35e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:16.489433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92530749-2ba0-4b21-961a-5f1ce6e59f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:16.490091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92530749-2ba0-4b21-961a-5f1ce6e59f40 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:16.490374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92530749-2ba0-4b21-961a-5f1ce6e59f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:16.490657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92530749-2ba0-4b21-961a-5f1ce6e59f40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:16.491496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92530749-2ba0-4b21-961a-5f1ce6e59f40 None admin] RBAC: Adding query filter params (group.id=e0162af404b44cd5b4d8886df243dac7, role.id=c465d586f0ec4b59815b23087d757fd2, scope.project.id=c5de3bc9e442415487283bf9414ae97d) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:16.491805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92530749-2ba0-4b21-961a-5f1ce6e59f40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:16.493523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92530749-2ba0-4b21-961a-5f1ce6e59f40 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:16.499733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 223/451] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Jul 18 16:30:16 2022] GET /identity/v3/role_assignments?group.id=e0162af404b44cd5b4d8886df243dac7&scope.project.id=c5de3bc9e442415487283bf9414ae97d&role.id=c465d586f0ec4b59815b23087d757fd2 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:16.500127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe4d83-1216-4946-bbe3-a63d4c3d460c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:16.501681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe4d83-1216-4946-bbe3-a63d4c3d460c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:16.534211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:16.614991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 229/452] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:30:16.621237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92530749-2ba0-4b21-961a-5f1ce6e59f40 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:16.712838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:16.731183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92530749-2ba0-4b21-961a-5f1ce6e59f40 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:16.770081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce42408-f478-4094-ad1c-ddb10facdd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:16.770959 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce42408-f478-4094-ad1c-ddb10facdd55 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:16.771240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce42408-f478-4094-ad1c-ddb10facdd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:16.773085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce42408-f478-4094-ad1c-ddb10facdd55 None admin] PATH_INFO: `/v3/projects/fb46b30ed74742b9a95d61944ef39381` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:16.773085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce42408-f478-4094-ad1c-ddb10facdd55 None admin] RBAC: Authorizing `identity:get_project(project_id=fb46b30ed74742b9a95d61944ef39381)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:16.774192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce42408-f478-4094-ad1c-ddb10facdd55 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:16.775876 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 224/453] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:30:16 2022] GET /identity/v3/projects/fb46b30ed74742b9a95d61944ef39381 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:16.781524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe4d83-1216-4946-bbe3-a63d4c3d460c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:16.888820 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe4d83-1216-4946-bbe3-a63d4c3d460c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:16.915114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:16.923103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bbcd5-72d9-4c65-ab43-3548d5d61d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:16.923976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bbcd5-72d9-4c65-ab43-3548d5d61d58 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:16.924255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bbcd5-72d9-4c65-ab43-3548d5d61d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:16.924526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bbcd5-72d9-4c65-ab43-3548d5d61d58 None admin] PATH_INFO: `/v3/groups/356b06a90e2e4ebcadc8fde30d2b0dc5` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:16.925729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bbcd5-72d9-4c65-ab43-3548d5d61d58 None admin] RBAC: Authorizing `identity:get_group(group_id=356b06a90e2e4ebcadc8fde30d2b0dc5)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:16.930707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bbcd5-72d9-4c65-ab43-3548d5d61d58 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:16.930707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 230/454] 10.222.0.88 () {56 vars in 1180 bytes} [Mon Jul 18 16:30:16 2022] GET /identity/v3/groups/356b06a90e2e4ebcadc8fde30d2b0dc5 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:16.934571 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce42408-f478-4094-ad1c-ddb10facdd55 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:17.036035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce42408-f478-4094-ad1c-ddb10facdd55 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:17.070878 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a4071-b3af-4a3b-8b8e-f583c403d224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:17.071530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a4071-b3af-4a3b-8b8e-f583c403d224 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:17.071814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a4071-b3af-4a3b-8b8e-f583c403d224 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:17.072093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a4071-b3af-4a3b-8b8e-f583c403d224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:17.072948 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a4071-b3af-4a3b-8b8e-f583c403d224 None admin] RBAC: Adding query filter params (group.id=356b06a90e2e4ebcadc8fde30d2b0dc5, role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=fb46b30ed74742b9a95d61944ef39381) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:17.073263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a4071-b3af-4a3b-8b8e-f583c403d224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:17.074705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a4071-b3af-4a3b-8b8e-f583c403d224 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:17.080930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 225/455] 10.222.0.88 () {56 vars in 1376 bytes} [Mon Jul 18 16:30:16 2022] GET /identity/v3/role_assignments?group.id=356b06a90e2e4ebcadc8fde30d2b0dc5&scope.project.id=fb46b30ed74742b9a95d61944ef39381&role.id=81457691f3a2456fa33358cb6a922d05 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:19.532294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee58a4-fe42-4bf9-a0f6-11b08ae9ee94 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:19.532294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee58a4-fe42-4bf9-a0f6-11b08ae9ee94 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:19.532294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee58a4-fe42-4bf9-a0f6-11b08ae9ee94 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:19.533777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 231/456] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:19.539484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81914308-2e11-4384-ad1d-e0d5f3b86fdb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:19.539968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81914308-2e11-4384-ad1d-e0d5f3b86fdb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:19.540044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81914308-2e11-4384-ad1d-e0d5f3b86fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:19.574134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-81914308-2e11-4384-ad1d-e0d5f3b86fdb None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:19.575374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-81914308-2e11-4384-ad1d-e0d5f3b86fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:19.589191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:19.714679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 226/457] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:19.765206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee58a4-fe42-4bf9-a0f6-11b08ae9ee94 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:19.990723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e94bf-b4dc-4b7a-b050-ac893bea916e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:19.991054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e94bf-b4dc-4b7a-b050-ac893bea916e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:19.991286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e94bf-b4dc-4b7a-b050-ac893bea916e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:19.993149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 227/458] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:20.006421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45d60-3d3b-4574-a1d5-6f73c90d3718 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:20.006685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45d60-3d3b-4574-a1d5-6f73c90d3718 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:20.008345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45d60-3d3b-4574-a1d5-6f73c90d3718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:20.045066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-06e45d60-3d3b-4574-a1d5-6f73c90d3718 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:20.045066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-06e45d60-3d3b-4574-a1d5-6f73c90d3718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:20.049776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee58a4-fe42-4bf9-a0f6-11b08ae9ee94 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:20.157695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4211bb-7256-463f-a33e-c41323893d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:20.158890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4211bb-7256-463f-a33e-c41323893d66 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:20.159402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4211bb-7256-463f-a33e-c41323893d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:20.159900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4211bb-7256-463f-a33e-c41323893d66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:20.189755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4211bb-7256-463f-a33e-c41323893d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:20.191557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4211bb-7256-463f-a33e-c41323893d66 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2b4211bb-7256-463f-a33e-c41323893d66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:20.234225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:20.235206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:20.236009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:20.236009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:20.236009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:20.236009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:20.246960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 232/459] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 483 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:20.249808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636a9aa-9030-45a7-8ddd-25ec0e5f3cc0 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:20.250311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636a9aa-9030-45a7-8ddd-25ec0e5f3cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:20.250808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636a9aa-9030-45a7-8ddd-25ec0e5f3cc0 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:20.252568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 233/460] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:20.257312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636a9aa-9030-45a7-8ddd-25ec0e5f3cc0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:20.317974 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 228/461] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:20.325154 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:20.329652 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae3256-f740-48f5-a79d-79e903722aef None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:20.330491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae3256-f740-48f5-a79d-79e903722aef None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:20.332928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae3256-f740-48f5-a79d-79e903722aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:20.358522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6fae3256-f740-48f5-a79d-79e903722aef None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:20.360074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6fae3256-f740-48f5-a79d-79e903722aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:20.395737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636a9aa-9030-45a7-8ddd-25ec0e5f3cc0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:20.446216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 229/462] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:20.447868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0290b2f-a802-4488-a3ae-946a8e61c92a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:20.448298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0290b2f-a802-4488-a3ae-946a8e61c92a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:20.448713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0290b2f-a802-4488-a3ae-946a8e61c92a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:20.450504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 230/463] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:20.451920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0290b2f-a802-4488-a3ae-946a8e61c92a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:20.460628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:20.483839 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77569237-f663-4018-95ad-557917f4ca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:20.484522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77569237-f663-4018-95ad-557917f4ca1e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:20.484805 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77569237-f663-4018-95ad-557917f4ca1e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:20.485126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77569237-f663-4018-95ad-557917f4ca1e None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:20.485885 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77569237-f663-4018-95ad-557917f4ca1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:20.486175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77569237-f663-4018-95ad-557917f4ca1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:20.487676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77569237-f663-4018-95ad-557917f4ca1e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:20.509941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 234/464] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:20.511429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23d6d6-9ead-4a50-8090-bbf8a1443c5a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:20.511730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23d6d6-9ead-4a50-8090-bbf8a1443c5a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:20.512023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23d6d6-9ead-4a50-8090-bbf8a1443c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:20.530619 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7c23d6d6-9ead-4a50-8090-bbf8a1443c5a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:20.531913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7c23d6d6-9ead-4a50-8090-bbf8a1443c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:20.595216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0290b2f-a802-4488-a3ae-946a8e61c92a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:20.617049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 235/465] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:20.618244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23d6d6-9ead-4a50-8090-bbf8a1443c5a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:20.645416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feec338-f01e-4cd9-bbf1-dfc4bd0971c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:20.646221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feec338-f01e-4cd9-bbf1-dfc4bd0971c9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:20.646504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feec338-f01e-4cd9-bbf1-dfc4bd0971c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:20.646780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feec338-f01e-4cd9-bbf1-dfc4bd0971c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:20.659315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feec338-f01e-4cd9-bbf1-dfc4bd0971c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:20.660772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feec338-f01e-4cd9-bbf1-dfc4bd0971c9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:20.679912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1feec338-f01e-4cd9-bbf1-dfc4bd0971c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:20.683264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:20.684128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:20.685037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:20.685037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:20.685037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:20.685037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:20.691639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 231/466] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:20.691639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feec338-f01e-4cd9-bbf1-dfc4bd0971c9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:20.752869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23d6d6-9ead-4a50-8090-bbf8a1443c5a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:20.768017 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:20.790976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87221da-32a5-4f35-9949-32eacc3484b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:20.791785 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87221da-32a5-4f35-9949-32eacc3484b5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:20.792063 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87221da-32a5-4f35-9949-32eacc3484b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:20.792332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87221da-32a5-4f35-9949-32eacc3484b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:20.805193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87221da-32a5-4f35-9949-32eacc3484b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:20.806959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87221da-32a5-4f35-9949-32eacc3484b5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:20.814930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feec338-f01e-4cd9-bbf1-dfc4bd0971c9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c87221da-32a5-4f35-9949-32eacc3484b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:20.826114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:20.826949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:20.827796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:20.827796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:20.827796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:20.827796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:20.832276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 236/467] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:20.833190 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87221da-32a5-4f35-9949-32eacc3484b5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:20.858781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b03e5-bda0-4842-9382-47a2ef911897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:20.859577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b03e5-bda0-4842-9382-47a2ef911897 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:20.859931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b03e5-bda0-4842-9382-47a2ef911897 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:20.860275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b03e5-bda0-4842-9382-47a2ef911897 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:20.861188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b03e5-bda0-4842-9382-47a2ef911897 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:20.862814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b03e5-bda0-4842-9382-47a2ef911897 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:20.934655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:20.945912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87221da-32a5-4f35-9949-32eacc3484b5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:20.965016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 232/468] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:30:20 2022] POST /identity/v3/users => generated 246 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:30:20.969072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b03e5-bda0-4842-9382-47a2ef911897 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:21.011308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66d8af-3f13-463b-9934-5587109a3451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:21.012108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66d8af-3f13-463b-9934-5587109a3451 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:21.012385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66d8af-3f13-463b-9934-5587109a3451 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:21.012657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66d8af-3f13-463b-9934-5587109a3451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:21.029753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66d8af-3f13-463b-9934-5587109a3451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:21.032025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66d8af-3f13-463b-9934-5587109a3451 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2e66d8af-3f13-463b-9934-5587109a3451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:21.060875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:21.061795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:21.062665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:21.062665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:21.062665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:21.062665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:21.062665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 237/469] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:21.063705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66d8af-3f13-463b-9934-5587109a3451 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:21.233681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b03e5-bda0-4842-9382-47a2ef911897 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:21.254219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:21.255702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66d8af-3f13-463b-9934-5587109a3451 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:21.291660 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9c73b-21d0-4154-bf54-9d89b11065bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:21.292346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9c73b-21d0-4154-bf54-9d89b11065bb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:21.292634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9c73b-21d0-4154-bf54-9d89b11065bb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:21.292928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9c73b-21d0-4154-bf54-9d89b11065bb None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:21.293714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9c73b-21d0-4154-bf54-9d89b11065bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:21.293998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9c73b-21d0-4154-bf54-9d89b11065bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:21.295870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9c73b-21d0-4154-bf54-9d89b11065bb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:21.303419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7ec2c-3a45-4cf5-88b8-3b2f1a1f7d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:21.304103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7ec2c-3a45-4cf5-88b8-3b2f1a1f7d01 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:21.304382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7ec2c-3a45-4cf5-88b8-3b2f1a1f7d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:21.304661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7ec2c-3a45-4cf5-88b8-3b2f1a1f7d01 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:21.305440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7ec2c-3a45-4cf5-88b8-3b2f1a1f7d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:21.305751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7ec2c-3a45-4cf5-88b8-3b2f1a1f7d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:21.307156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7ec2c-3a45-4cf5-88b8-3b2f1a1f7d01 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:21.320470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 233/470] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:21.321791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9c73b-21d0-4154-bf54-9d89b11065bb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:21.336140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 238/471] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:21.337536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db2771-8fa4-47d6-b9c0-61c911b68c18 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:21.337832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db2771-8fa4-47d6-b9c0-61c911b68c18 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:21.338116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db2771-8fa4-47d6-b9c0-61c911b68c18 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:21.340355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 239/472] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:21.342284 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db2771-8fa4-47d6-b9c0-61c911b68c18 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:21.346071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:21.486464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db2771-8fa4-47d6-b9c0-61c911b68c18 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:21.503054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9c73b-21d0-4154-bf54-9d89b11065bb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:21.543743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584cbec-6ae0-4ef3-93bb-1625c4b123a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:21.545909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584cbec-6ae0-4ef3-93bb-1625c4b123a9 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:21.546597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584cbec-6ae0-4ef3-93bb-1625c4b123a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:21.547126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584cbec-6ae0-4ef3-93bb-1625c4b123a9 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:21.548836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584cbec-6ae0-4ef3-93bb-1625c4b123a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:21.550505 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dbdd8-1898-4ed7-9d85-667847facd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:21.551367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dbdd8-1898-4ed7-9d85-667847facd5d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:21.551834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dbdd8-1898-4ed7-9d85-667847facd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:21.552107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584cbec-6ae0-4ef3-93bb-1625c4b123a9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:21.552481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dbdd8-1898-4ed7-9d85-667847facd5d None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:21.553427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dbdd8-1898-4ed7-9d85-667847facd5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:21.553909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dbdd8-1898-4ed7-9d85-667847facd5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:21.555455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dbdd8-1898-4ed7-9d85-667847facd5d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:21.572928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 234/473] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:21.572928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dbdd8-1898-4ed7-9d85-667847facd5d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:21.603901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:21.662982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 240/474] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:30:21 2022] POST /identity/v3/users => generated 248 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:30:21.664125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2f58d-c1a3-435c-9745-931be87d0d5a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:21.664418 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2f58d-c1a3-435c-9745-931be87d0d5a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:21.664730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2f58d-c1a3-435c-9745-931be87d0d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:21.683746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-19b2f58d-c1a3-435c-9745-931be87d0d5a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:21.684969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-19b2f58d-c1a3-435c-9745-931be87d0d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:21.691968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dbdd8-1898-4ed7-9d85-667847facd5d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:21.742628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677b264-5d38-4a44-9f94-3e16c3a384db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:21.744041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677b264-5d38-4a44-9f94-3e16c3a384db None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:21.744551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677b264-5d38-4a44-9f94-3e16c3a384db None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:21.745062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677b264-5d38-4a44-9f94-3e16c3a384db None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:21.746747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677b264-5d38-4a44-9f94-3e16c3a384db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:21.749405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677b264-5d38-4a44-9f94-3e16c3a384db None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:21.754447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:21.771448 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 241/475] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:21.772159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2f58d-c1a3-435c-9745-931be87d0d5a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:21.865285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 235/476] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:30:21 2022] POST /identity/v3/users => generated 249 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:30:21.865285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677b264-5d38-4a44-9f94-3e16c3a384db None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:21.972653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2f58d-c1a3-435c-9745-931be87d0d5a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:22.011818 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:22.027394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52302b49-0b57-45cc-95fe-6601810826fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:22.028100 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52302b49-0b57-45cc-95fe-6601810826fe None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:22.028381 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52302b49-0b57-45cc-95fe-6601810826fe None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:22.028659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52302b49-0b57-45cc-95fe-6601810826fe None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:22.029710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52302b49-0b57-45cc-95fe-6601810826fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:22.032107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52302b49-0b57-45cc-95fe-6601810826fe None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:22.033804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677b264-5d38-4a44-9f94-3e16c3a384db None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:22.085130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0d76f-a889-455c-8167-dee641bf9547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:22.085824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0d76f-a889-455c-8167-dee641bf9547 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:22.086108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0d76f-a889-455c-8167-dee641bf9547 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:22.086386 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0d76f-a889-455c-8167-dee641bf9547 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:22.087151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0d76f-a889-455c-8167-dee641bf9547 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:22.088621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0d76f-a889-455c-8167-dee641bf9547 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:22.120053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 242/477] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:30:21 2022] POST /identity/v3/users => generated 248 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:30:22.121930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.sql.core [None req-f8e0d76f-a889-455c-8167-dee641bf9547 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 16:30:22.121930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 16:30:22.121930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [parameters: {'id': '56817945ddf84d35980ab48fcecf9597', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 16:30:22.121930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80522) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f8e0d76f-a889-455c-8167-dee641bf9547 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result.read() Jul 18 16:30:22.139760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:30:22.140718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application yield resource Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise self.value Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application session.commit() Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 16:30:22.142207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.session.flush() Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application compat.raise_( Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 16:30:22.143326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:30:22.144211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result.read() Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:30:22.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [parameters: {'id': '56817945ddf84d35980ab48fcecf9597', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:22.147890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 16:30:22.150096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:22.157328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 236/478] 10.222.0.88 () {60 vars in 1131 bytes} [Mon Jul 18 16:30:21 2022] POST /identity/v3/roles => generated 144 bytes in 280 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:22.157488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52302b49-0b57-45cc-95fe-6601810826fe None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:22.204147 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:22.271779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52302b49-0b57-45cc-95fe-6601810826fe None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:22.311977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaa821-f74e-43fe-91f0-939d3ddeac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:22.312813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa821-f74e-43fe-91f0-939d3ddeac94 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:22.313132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa821-f74e-43fe-91f0-939d3ddeac94 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:22.313434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa821-f74e-43fe-91f0-939d3ddeac94 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:22.326395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaa821-f74e-43fe-91f0-939d3ddeac94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-edaaa821-f74e-43fe-91f0-939d3ddeac94 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:22.334896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:22.337137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:22.339148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 243/479] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:30:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:22.342909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0d76f-a889-455c-8167-dee641bf9547 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:22.458957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0d76f-a889-455c-8167-dee641bf9547 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:22.502654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ffbd60-756a-4798-bf99-3ba3bb03d2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:22.503691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffbd60-756a-4798-bf99-3ba3bb03d2bb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:22.504178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffbd60-756a-4798-bf99-3ba3bb03d2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:22.504652 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ffbd60-756a-4798-bf99-3ba3bb03d2bb None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:22.505658 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffbd60-756a-4798-bf99-3ba3bb03d2bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:22.506167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffbd60-756a-4798-bf99-3ba3bb03d2bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:22.507756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ffbd60-756a-4798-bf99-3ba3bb03d2bb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:22.520261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 237/480] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:23.224262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea8ec6-d315-4c50-9f0c-7c1bf3eadad7 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:23.224739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea8ec6-d315-4c50-9f0c-7c1bf3eadad7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:23.224739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea8ec6-d315-4c50-9f0c-7c1bf3eadad7 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:23.226476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 244/481] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:23.232231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18675124-3979-46db-8bf6-094d4332f978 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:23.232674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18675124-3979-46db-8bf6-094d4332f978 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:23.232900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18675124-3979-46db-8bf6-094d4332f978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:23.261913 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-18675124-3979-46db-8bf6-094d4332f978 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:23.263180 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-18675124-3979-46db-8bf6-094d4332f978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:23.279014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:23.373777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 238/482] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:23.408671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea8ec6-d315-4c50-9f0c-7c1bf3eadad7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:23.630918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea8ec6-d315-4c50-9f0c-7c1bf3eadad7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:23.633387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:23.670966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01127860-e639-40a0-812c-cdfd77baad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:23.672793 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01127860-e639-40a0-812c-cdfd77baad6b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:23.673217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01127860-e639-40a0-812c-cdfd77baad6b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:23.676912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01127860-e639-40a0-812c-cdfd77baad6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:23.700909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01127860-e639-40a0-812c-cdfd77baad6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-01127860-e639-40a0-812c-cdfd77baad6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:23.715830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:23.717713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:23.719062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:23.721554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 245/483] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:23.734027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18675124-3979-46db-8bf6-094d4332f978 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:23.923850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18675124-3979-46db-8bf6-094d4332f978 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:23.986565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad54fa5-b65a-44a7-844f-0a5ecacbbcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:23.987256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54fa5-b65a-44a7-844f-0a5ecacbbcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:23.987541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54fa5-b65a-44a7-844f-0a5ecacbbcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:23.987818 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54fa5-b65a-44a7-844f-0a5ecacbbcb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:23.988578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad54fa5-b65a-44a7-844f-0a5ecacbbcb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:23.989103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad54fa5-b65a-44a7-844f-0a5ecacbbcb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:23.990486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad54fa5-b65a-44a7-844f-0a5ecacbbcb1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:24.005396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 239/484] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:24.010584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01127860-e639-40a0-812c-cdfd77baad6b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:24.195977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01127860-e639-40a0-812c-cdfd77baad6b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:24.241275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778352c-2be3-44d6-8d58-dce4ca1d18ec None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.241769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778352c-2be3-44d6-8d58-dce4ca1d18ec None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.241870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778352c-2be3-44d6-8d58-dce4ca1d18ec None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.245401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 240/485] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:24.249760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276978fe-c3f0-473e-aedc-70561074e744 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.250085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276978fe-c3f0-473e-aedc-70561074e744 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.250329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276978fe-c3f0-473e-aedc-70561074e744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.251512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5025f-1569-48a1-bca7-763b89447af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:24.252862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5025f-1569-48a1-bca7-763b89447af8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.253396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5025f-1569-48a1-bca7-763b89447af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.253895 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5025f-1569-48a1-bca7-763b89447af8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.266549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5025f-1569-48a1-bca7-763b89447af8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:24.268453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5025f-1569-48a1-bca7-763b89447af8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:24.289303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:24.290880 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-276978fe-c3f0-473e-aedc-70561074e744 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:24.292117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-276978fe-c3f0-473e-aedc-70561074e744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-add5025f-1569-48a1-bca7-763b89447af8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:24.302972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:24.303884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:24.304707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:24.304707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:24.304707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:24.304707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:24.305628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 246/486] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:24.312783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5025f-1569-48a1-bca7-763b89447af8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:24.344858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:24.411469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 241/487] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:24.436274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276978fe-c3f0-473e-aedc-70561074e744 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:24.484784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5025f-1569-48a1-bca7-763b89447af8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:24.528645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41d9c8-9c76-4a30-b6f5-bdf3c2d79fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:24.529395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41d9c8-9c76-4a30-b6f5-bdf3c2d79fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.529680 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41d9c8-9c76-4a30-b6f5-bdf3c2d79fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.529961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41d9c8-9c76-4a30-b6f5-bdf3c2d79fb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.530699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41d9c8-9c76-4a30-b6f5-bdf3c2d79fb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:24.531038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41d9c8-9c76-4a30-b6f5-bdf3c2d79fb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:24.532416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41d9c8-9c76-4a30-b6f5-bdf3c2d79fb4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:24.545186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 247/488] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:24.552906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41d9c8-9c76-4a30-b6f5-bdf3c2d79fb4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:24.673338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:24.677065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276978fe-c3f0-473e-aedc-70561074e744 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:24.708070 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41d9c8-9c76-4a30-b6f5-bdf3c2d79fb4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:24.721773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817c274-a538-4154-8ade-23f5c607640a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:24.723053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817c274-a538-4154-8ade-23f5c607640a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.723562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817c274-a538-4154-8ade-23f5c607640a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.724030 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817c274-a538-4154-8ade-23f5c607640a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.727857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:24.741206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817c274-a538-4154-8ade-23f5c607640a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2817c274-a538-4154-8ade-23f5c607640a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:24.747317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:24.748242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:24.749707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:24.750311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb811d3-9717-4c67-9aeb-dde11d3d723b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:24.751425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb811d3-9717-4c67-9aeb-dde11d3d723b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.751707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb811d3-9717-4c67-9aeb-dde11d3d723b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.751980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb811d3-9717-4c67-9aeb-dde11d3d723b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.753316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 242/489] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:24.755220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190493f5-3256-4467-8ebf-ebcdea17a92e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.755737 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190493f5-3256-4467-8ebf-ebcdea17a92e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.756218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190493f5-3256-4467-8ebf-ebcdea17a92e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.762027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb811d3-9717-4c67-9aeb-dde11d3d723b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:24.762027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb811d3-9717-4c67-9aeb-dde11d3d723b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:24.762443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 243/490] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:24.764080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190493f5-3256-4467-8ebf-ebcdea17a92e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-efb811d3-9717-4c67-9aeb-dde11d3d723b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:24.788529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:24.789716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:24.790613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:24.790613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:24.790613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:24.790613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:24.790613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:24.790613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:24.790613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:24.790613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:24.790613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:30:24.790613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:24.795666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 248/491] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Jul 18 16:30:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:24.804194 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6110f-0bc0-46c4-9cfc-1587dc1e410a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.804527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6110f-0bc0-46c4-9cfc-1587dc1e410a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.804817 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6110f-0bc0-46c4-9cfc-1587dc1e410a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.831323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a5c6110f-0bc0-46c4-9cfc-1587dc1e410a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:24.833041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a5c6110f-0bc0-46c4-9cfc-1587dc1e410a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:24.906187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190493f5-3256-4467-8ebf-ebcdea17a92e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:24.918064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 249/492] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:24.919252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6110f-0bc0-46c4-9cfc-1587dc1e410a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:24.949615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61573365-1d61-42a4-937a-1a048f3df580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:24.950384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61573365-1d61-42a4-937a-1a048f3df580 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.950811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61573365-1d61-42a4-937a-1a048f3df580 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.951097 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61573365-1d61-42a4-937a-1a048f3df580 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.951860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61573365-1d61-42a4-937a-1a048f3df580 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:24.952168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61573365-1d61-42a4-937a-1a048f3df580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:24.953597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61573365-1d61-42a4-937a-1a048f3df580 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:24.962326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 244/493] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:24.963581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fae283-f853-4045-8fd5-ce4e9101b341 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:24.963878 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fae283-f853-4045-8fd5-ce4e9101b341 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:24.964160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fae283-f853-4045-8fd5-ce4e9101b341 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:24.969011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 245/494] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:24.969011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fae283-f853-4045-8fd5-ce4e9101b341 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:25.007602 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:25.038809 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:25.043865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6110f-0bc0-46c4-9cfc-1587dc1e410a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:25.081249 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4680fc9-8dbc-4b90-a7be-caf4c5c0726f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:25.081920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4680fc9-8dbc-4b90-a7be-caf4c5c0726f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.082200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4680fc9-8dbc-4b90-a7be-caf4c5c0726f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.082480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4680fc9-8dbc-4b90-a7be-caf4c5c0726f None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.083306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4680fc9-8dbc-4b90-a7be-caf4c5c0726f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:25.083616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4680fc9-8dbc-4b90-a7be-caf4c5c0726f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:25.085098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4680fc9-8dbc-4b90-a7be-caf4c5c0726f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:25.086467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fae283-f853-4045-8fd5-ce4e9101b341 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:25.132336 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248c928-c8c3-4558-a528-35e2a0237766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:25.133262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248c928-c8c3-4558-a528-35e2a0237766 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.133533 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248c928-c8c3-4558-a528-35e2a0237766 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.133831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248c928-c8c3-4558-a528-35e2a0237766 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.136232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 250/495] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Jul 18 16:30:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:25.141951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4680fc9-8dbc-4b90-a7be-caf4c5c0726f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:25.147837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248c928-c8c3-4558-a528-35e2a0237766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4248c928-c8c3-4558-a528-35e2a0237766 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:25.156471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:25.159353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 246/496] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:25.160217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831279e-2425-40a3-b98b-e0393b9e5c3e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.161158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831279e-2425-40a3-b98b-e0393b9e5c3e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.161158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831279e-2425-40a3-b98b-e0393b9e5c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.178417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7831279e-2425-40a3-b98b-e0393b9e5c3e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:25.180160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7831279e-2425-40a3-b98b-e0393b9e5c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:25.265063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 247/497] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:25.265834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831279e-2425-40a3-b98b-e0393b9e5c3e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:25.279916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4680fc9-8dbc-4b90-a7be-caf4c5c0726f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:25.307394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:25.318655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec802dab-5f0a-4748-a392-622a960459cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:25.319456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec802dab-5f0a-4748-a392-622a960459cd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.319739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec802dab-5f0a-4748-a392-622a960459cd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.324722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec802dab-5f0a-4748-a392-622a960459cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.332825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec802dab-5f0a-4748-a392-622a960459cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:25.334358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec802dab-5f0a-4748-a392-622a960459cd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:25.337131 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ec802dab-5f0a-4748-a392-622a960459cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:25.352725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:25.353938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:25.354791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:25.354791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:25.354791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:25.354791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:25.355191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 251/498] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:25.356063 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec802dab-5f0a-4748-a392-622a960459cd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:25.387138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831279e-2425-40a3-b98b-e0393b9e5c3e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:25.422008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e249b89-4df7-4245-b5b5-25b3668d1d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:25.422665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e249b89-4df7-4245-b5b5-25b3668d1d68 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.422949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e249b89-4df7-4245-b5b5-25b3668d1d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.423224 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e249b89-4df7-4245-b5b5-25b3668d1d68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.459776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec802dab-5f0a-4748-a392-622a960459cd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:25.494080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e249b89-4df7-4245-b5b5-25b3668d1d68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:25.495550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e249b89-4df7-4245-b5b5-25b3668d1d68 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:25.496917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5b7d9-3c60-47fb-bde8-40bdb992b0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:25.497638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5b7d9-3c60-47fb-bde8-40bdb992b0cc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.497922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5b7d9-3c60-47fb-bde8-40bdb992b0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.498208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5b7d9-3c60-47fb-bde8-40bdb992b0cc None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.498974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5b7d9-3c60-47fb-bde8-40bdb992b0cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:25.499283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5b7d9-3c60-47fb-bde8-40bdb992b0cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:25.500672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5b7d9-3c60-47fb-bde8-40bdb992b0cc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:25.509262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 252/499] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:25.509908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5b7d9-3c60-47fb-bde8-40bdb992b0cc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:25.549107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:25.553892 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:25.601568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 248/500] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:25.602930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336e8e6-6d53-4d4b-ac10-08d3c6556f41 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.603221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336e8e6-6d53-4d4b-ac10-08d3c6556f41 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.603504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336e8e6-6d53-4d4b-ac10-08d3c6556f41 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.604936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 249/501] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:25.605529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336e8e6-6d53-4d4b-ac10-08d3c6556f41 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:25.616464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5b7d9-3c60-47fb-bde8-40bdb992b0cc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:25.651404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5807bf5-f790-4fc0-a7e1-3779bd54f1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:25.652212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5807bf5-f790-4fc0-a7e1-3779bd54f1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.652493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5807bf5-f790-4fc0-a7e1-3779bd54f1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.652772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5807bf5-f790-4fc0-a7e1-3779bd54f1f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.668102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5807bf5-f790-4fc0-a7e1-3779bd54f1f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e5807bf5-f790-4fc0-a7e1-3779bd54f1f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:25.676483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:25.677437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:25.678217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 253/502] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:25.678864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5807bf5-f790-4fc0-a7e1-3779bd54f1f5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:25.718032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336e8e6-6d53-4d4b-ac10-08d3c6556f41 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:25.753782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c428147-a363-4caa-8e5e-df607116457a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:25.754822 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428147-a363-4caa-8e5e-df607116457a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.755383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428147-a363-4caa-8e5e-df607116457a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.755862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c428147-a363-4caa-8e5e-df607116457a None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.756976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c428147-a363-4caa-8e5e-df607116457a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:25.757498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c428147-a363-4caa-8e5e-df607116457a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:25.759708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c428147-a363-4caa-8e5e-df607116457a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:25.771720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 250/503] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:25.776265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c428147-a363-4caa-8e5e-df607116457a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:25.781757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:25.785293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:25.785608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5807bf5-f790-4fc0-a7e1-3779bd54f1f5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:25.819790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023138be-0912-49fb-9d9c-51962ff25554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:25.820506 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023138be-0912-49fb-9d9c-51962ff25554 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.820792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023138be-0912-49fb-9d9c-51962ff25554 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.821110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023138be-0912-49fb-9d9c-51962ff25554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.833589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023138be-0912-49fb-9d9c-51962ff25554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:25.835062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023138be-0912-49fb-9d9c-51962ff25554 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-023138be-0912-49fb-9d9c-51962ff25554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:25.852349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:25.858132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:25.858965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:25.858965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:25.858965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:25.858965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:25.858965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 254/504] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:25.858965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c9f07-4728-44df-9f97-e1fda28d8b01 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.858965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c9f07-4728-44df-9f97-e1fda28d8b01 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.858965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c9f07-4728-44df-9f97-e1fda28d8b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.873651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8a2c9f07-4728-44df-9f97-e1fda28d8b01 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:25.874849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8a2c9f07-4728-44df-9f97-e1fda28d8b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:25.876166 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c428147-a363-4caa-8e5e-df607116457a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:25.910990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19938003-3fb8-4cc6-bbbd-1fba2ec495dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:25.911792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19938003-3fb8-4cc6-bbbd-1fba2ec495dd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:25.912077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19938003-3fb8-4cc6-bbbd-1fba2ec495dd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:25.912350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19938003-3fb8-4cc6-bbbd-1fba2ec495dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:25.923849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19938003-3fb8-4cc6-bbbd-1fba2ec495dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:25.925368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19938003-3fb8-4cc6-bbbd-1fba2ec495dd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:25.946998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 255/505] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-19938003-3fb8-4cc6-bbbd-1fba2ec495dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:25.947274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:25.948132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:25.953774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:25.953774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:25.953774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:25.953774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:25.954007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c9f07-4728-44df-9f97-e1fda28d8b01 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:25.954076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 251/506] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:25.954076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19938003-3fb8-4cc6-bbbd-1fba2ec495dd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.019149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:26.035886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:26.055020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c9f07-4728-44df-9f97-e1fda28d8b01 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.061387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19938003-3fb8-4cc6-bbbd-1fba2ec495dd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.090831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660fc8a-fe6b-4179-bb41-1a3280c12dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.091982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660fc8a-fe6b-4179-bb41-1a3280c12dcc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.092259 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660fc8a-fe6b-4179-bb41-1a3280c12dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.092542 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660fc8a-fe6b-4179-bb41-1a3280c12dcc None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.093334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660fc8a-fe6b-4179-bb41-1a3280c12dcc None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:26.093640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660fc8a-fe6b-4179-bb41-1a3280c12dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:26.095025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8660fc8a-fe6b-4179-bb41-1a3280c12dcc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:26.097567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de86eaa-7fa7-45e9-b9b8-a150333179f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.098290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de86eaa-7fa7-45e9-b9b8-a150333179f3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.098573 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de86eaa-7fa7-45e9-b9b8-a150333179f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.098847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de86eaa-7fa7-45e9-b9b8-a150333179f3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.103179 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 256/507] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:26.103978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660fc8a-fe6b-4179-bb41-1a3280c12dcc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.106724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de86eaa-7fa7-45e9-b9b8-a150333179f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:26.108871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de86eaa-7fa7-45e9-b9b8-a150333179f3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2de86eaa-7fa7-45e9-b9b8-a150333179f3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:26.124998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:26.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 252/508] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:26.132141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de86eaa-7fa7-45e9-b9b8-a150333179f3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.212171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660fc8a-fe6b-4179-bb41-1a3280c12dcc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.233356 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de86eaa-7fa7-45e9-b9b8-a150333179f3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.247769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675d069-f78f-49b3-90f3-028172773756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.248439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675d069-f78f-49b3-90f3-028172773756 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.248725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675d069-f78f-49b3-90f3-028172773756 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.249035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675d069-f78f-49b3-90f3-028172773756 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.249766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675d069-f78f-49b3-90f3-028172773756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:26.250077 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675d069-f78f-49b3-90f3-028172773756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:26.251464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675d069-f78f-49b3-90f3-028172773756 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:26.253370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:26.264104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 257/509] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:26.265082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675d069-f78f-49b3-90f3-028172773756 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.267688 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c3d26-b0de-4eec-87a7-85ccf06bf551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.269703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:26.269805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c3d26-b0de-4eec-87a7-85ccf06bf551 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.269805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c3d26-b0de-4eec-87a7-85ccf06bf551 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.269805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c3d26-b0de-4eec-87a7-85ccf06bf551 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.269805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c3d26-b0de-4eec-87a7-85ccf06bf551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:26.270070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c3d26-b0de-4eec-87a7-85ccf06bf551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:26.271377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c3d26-b0de-4eec-87a7-85ccf06bf551 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:26.283729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 253/510] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:26.284525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c3d26-b0de-4eec-87a7-85ccf06bf551 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.373462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675d069-f78f-49b3-90f3-028172773756 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.385676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c3d26-b0de-4eec-87a7-85ccf06bf551 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.408501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2335961-ce40-49b5-99ce-006a9b1bb979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.409322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2335961-ce40-49b5-99ce-006a9b1bb979 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.409600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2335961-ce40-49b5-99ce-006a9b1bb979 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.409869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2335961-ce40-49b5-99ce-006a9b1bb979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.420300 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2335961-ce40-49b5-99ce-006a9b1bb979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:26.421888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2335961-ce40-49b5-99ce-006a9b1bb979 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:26.423372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5fa92-a793-4d0b-9fd8-a1a789601c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.424168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5fa92-a793-4d0b-9fd8-a1a789601c41 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.424458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5fa92-a793-4d0b-9fd8-a1a789601c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.424736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5fa92-a793-4d0b-9fd8-a1a789601c41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.435268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5fa92-a793-4d0b-9fd8-a1a789601c41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:26.436768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5fa92-a793-4d0b-9fd8-a1a789601c41 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d2335961-ce40-49b5-99ce-006a9b1bb979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:26.439857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:26.440812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:26.441715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:26.441715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:26.441715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:26.441715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:26.441715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 258/511] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:26.442282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2335961-ce40-49b5-99ce-006a9b1bb979 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5be5fa92-a793-4d0b-9fd8-a1a789601c41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:26.457552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:26.458439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:26.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:26.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:26.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:26.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:26.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 254/512] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:26.459674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5fa92-a793-4d0b-9fd8-a1a789601c41 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.492156 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:26.504045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:26.555157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2335961-ce40-49b5-99ce-006a9b1bb979 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.564139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5fa92-a793-4d0b-9fd8-a1a789601c41 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.596616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f3ef3-dc58-4d41-9351-d46f96876afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.597401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f3ef3-dc58-4d41-9351-d46f96876afd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.597752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f3ef3-dc58-4d41-9351-d46f96876afd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.598033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f3ef3-dc58-4d41-9351-d46f96876afd None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.598874 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f3ef3-dc58-4d41-9351-d46f96876afd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:26.599178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f3ef3-dc58-4d41-9351-d46f96876afd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:26.600626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f3ef3-dc58-4d41-9351-d46f96876afd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:26.604335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926fc08-76bf-477c-8b6c-32314ab10097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.605183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926fc08-76bf-477c-8b6c-32314ab10097 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.605479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926fc08-76bf-477c-8b6c-32314ab10097 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.605751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926fc08-76bf-477c-8b6c-32314ab10097 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.613159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926fc08-76bf-477c-8b6c-32314ab10097 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:26.614621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926fc08-76bf-477c-8b6c-32314ab10097 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:26.620146 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 259/513] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Jul 18 16:30:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:26.620320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f3ef3-dc58-4d41-9351-d46f96876afd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5926fc08-76bf-477c-8b6c-32314ab10097 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:26.631061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:26.632174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 255/514] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:26.638167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926fc08-76bf-477c-8b6c-32314ab10097 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.723632 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f3ef3-dc58-4d41-9351-d46f96876afd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.727565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:26.737524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:26.738577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926fc08-76bf-477c-8b6c-32314ab10097 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.758848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2b387-d3a1-428a-afc3-f9b6cccb3ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.759647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2b387-d3a1-428a-afc3-f9b6cccb3ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.759927 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2b387-d3a1-428a-afc3-f9b6cccb3ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.760202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2b387-d3a1-428a-afc3-f9b6cccb3ec3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.767640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2b387-d3a1-428a-afc3-f9b6cccb3ec3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:26.769146 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2b387-d3a1-428a-afc3-f9b6cccb3ec3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:26.773804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf681c07-f26c-468f-a4e5-1d5fefcebf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.778144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf681c07-f26c-468f-a4e5-1d5fefcebf44 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.778144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf681c07-f26c-468f-a4e5-1d5fefcebf44 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.778144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf681c07-f26c-468f-a4e5-1d5fefcebf44 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.778144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf681c07-f26c-468f-a4e5-1d5fefcebf44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:26.778144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf681c07-f26c-468f-a4e5-1d5fefcebf44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:26.778144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf681c07-f26c-468f-a4e5-1d5fefcebf44 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-0bf2b387-d3a1-428a-afc3-f9b6cccb3ec3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:26.783721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:26.784537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:26.785383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:26.785383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:26.785383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:26.785383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:26.785383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:26.785383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:26.785383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:26.785383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:26.785383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 260/515] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:26.785853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2b387-d3a1-428a-afc3-f9b6cccb3ec3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.790016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 256/516] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:26.790844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf681c07-f26c-468f-a4e5-1d5fefcebf44 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.897962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2b387-d3a1-428a-afc3-f9b6cccb3ec3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.910349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf681c07-f26c-468f-a4e5-1d5fefcebf44 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:26.933252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e14de-ff51-44a7-bb50-4f91f8ce7fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.933927 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e14de-ff51-44a7-bb50-4f91f8ce7fab None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.934209 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e14de-ff51-44a7-bb50-4f91f8ce7fab None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.934482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e14de-ff51-44a7-bb50-4f91f8ce7fab None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.935811 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e14de-ff51-44a7-bb50-4f91f8ce7fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:26.936925 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e14de-ff51-44a7-bb50-4f91f8ce7fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:26.938805 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e14de-ff51-44a7-bb50-4f91f8ce7fab None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:26.950056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 261/517] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:26.951751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e14de-ff51-44a7-bb50-4f91f8ce7fab None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:26.956386 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5dbebb-885a-497c-8bdf-66c9f9b4a772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:26.957084 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dbebb-885a-497c-8bdf-66c9f9b4a772 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:26.957377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dbebb-885a-497c-8bdf-66c9f9b4a772 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:26.957658 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5dbebb-885a-497c-8bdf-66c9f9b4a772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:26.979083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:26.981761 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:27.033906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5dbebb-885a-497c-8bdf-66c9f9b4a772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:27.035424 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5dbebb-885a-497c-8bdf-66c9f9b4a772 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:27.058409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e14de-ff51-44a7-bb50-4f91f8ce7fab None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:27.101473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bff1e-699e-436c-bc32-59ed3c629871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:27.102308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bff1e-699e-436c-bc32-59ed3c629871 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:27.102597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bff1e-699e-436c-bc32-59ed3c629871 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:27.102868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bff1e-699e-436c-bc32-59ed3c629871 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:27.103751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8bff1e-699e-436c-bc32-59ed3c629871 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:27.104068 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8bff1e-699e-436c-bc32-59ed3c629871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:27.105534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8bff1e-699e-436c-bc32-59ed3c629871 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:27.124063 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 262/518] 10.222.0.88 () {56 vars in 1139 bytes} [Mon Jul 18 16:30:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:27.125019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bff1e-699e-436c-bc32-59ed3c629871 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:27.145306 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 257/519] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:27.146640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5dbebb-885a-497c-8bdf-66c9f9b4a772 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:27.204590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:27.224482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:27.234571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bff1e-699e-436c-bc32-59ed3c629871 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:27.254860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5dbebb-885a-497c-8bdf-66c9f9b4a772 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:27.269270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7decd9-1f8b-46d8-9f99-9162c54b0a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:27.269934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7decd9-1f8b-46d8-9f99-9162c54b0a77 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:27.270212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7decd9-1f8b-46d8-9f99-9162c54b0a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:27.270488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7decd9-1f8b-46d8-9f99-9162c54b0a77 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:27.271472 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7decd9-1f8b-46d8-9f99-9162c54b0a77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:27.271782 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7decd9-1f8b-46d8-9f99-9162c54b0a77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:27.273447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7decd9-1f8b-46d8-9f99-9162c54b0a77 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:27.292955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee216da-ebd8-4467-aa8b-b389b0a57f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:27.293511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 263/520] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:27.293674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee216da-ebd8-4467-aa8b-b389b0a57f7f None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:27.293920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee216da-ebd8-4467-aa8b-b389b0a57f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:27.294073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7decd9-1f8b-46d8-9f99-9162c54b0a77 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:27.294761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee216da-ebd8-4467-aa8b-b389b0a57f7f None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:27.295441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee216da-ebd8-4467-aa8b-b389b0a57f7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:27.296899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee216da-ebd8-4467-aa8b-b389b0a57f7f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:27.402747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7decd9-1f8b-46d8-9f99-9162c54b0a77 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:27.408195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 258/521] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:30:27 2022] POST /identity/v3/users => generated 247 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:30:27.409081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee216da-ebd8-4467-aa8b-b389b0a57f7f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:27.453201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4772d48-e967-40bd-99f7-7df88fb3a763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:27.454134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4772d48-e967-40bd-99f7-7df88fb3a763 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:27.454303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4772d48-e967-40bd-99f7-7df88fb3a763 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:27.454583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4772d48-e967-40bd-99f7-7df88fb3a763 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:27.464107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4772d48-e967-40bd-99f7-7df88fb3a763 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:27.465671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4772d48-e967-40bd-99f7-7df88fb3a763 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a4772d48-e967-40bd-99f7-7df88fb3a763 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:27.484090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:27.485273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:27.486174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:27.486174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:27.486174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:27.486174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:27.486174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:27.486174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:27.486174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:27.486174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:27.486174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:30:27.486174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:27.486174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 264/522] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:27.487132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4772d48-e967-40bd-99f7-7df88fb3a763 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:27.501436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:27.539514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee216da-ebd8-4467-aa8b-b389b0a57f7f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:27.550737 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:27.583963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6e6cf-fb37-4c67-ac16-cec0eea56794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:27.584590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6e6cf-fb37-4c67-ac16-cec0eea56794 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:27.584875 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6e6cf-fb37-4c67-ac16-cec0eea56794 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:27.585547 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6e6cf-fb37-4c67-ac16-cec0eea56794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:27.613542 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4772d48-e967-40bd-99f7-7df88fb3a763 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:27.652214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7399d9-1ab9-45ae-a735-a0a15bfcbb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:27.653138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7399d9-1ab9-45ae-a735-a0a15bfcbb4e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:27.653550 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7399d9-1ab9-45ae-a735-a0a15bfcbb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:27.654391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7399d9-1ab9-45ae-a735-a0a15bfcbb4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:27.665860 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7399d9-1ab9-45ae-a735-a0a15bfcbb4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:27.667269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6e6cf-fb37-4c67-ac16-cec0eea56794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:27.667502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7399d9-1ab9-45ae-a735-a0a15bfcbb4e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:27.670015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6e6cf-fb37-4c67-ac16-cec0eea56794 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9c7399d9-1ab9-45ae-a735-a0a15bfcbb4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:27.687461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:27.694306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:27.695225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:27.695225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:27.695225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:27.695225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:27.695225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 265/523] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:27.695225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7399d9-1ab9-45ae-a735-a0a15bfcbb4e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:27.764232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:27.797826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 259/524] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:27.799410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6e6cf-fb37-4c67-ac16-cec0eea56794 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:27.813319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7399d9-1ab9-45ae-a735-a0a15bfcbb4e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:27.817424 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:27.852024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603b2e8-5984-44ce-9e4c-5b6ad7d0fc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:27.852684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b2e8-5984-44ce-9e4c-5b6ad7d0fc6a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:27.853014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b2e8-5984-44ce-9e4c-5b6ad7d0fc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:27.853297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603b2e8-5984-44ce-9e4c-5b6ad7d0fc6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:27.854171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603b2e8-5984-44ce-9e4c-5b6ad7d0fc6a None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=69ee145599384c6da2cba85a42832dcd) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:27.854486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603b2e8-5984-44ce-9e4c-5b6ad7d0fc6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:27.856411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603b2e8-5984-44ce-9e4c-5b6ad7d0fc6a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:27.865737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 266/525] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:27 2022] GET /identity/v3/role_assignments?user.id=69ee145599384c6da2cba85a42832dcd&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:27.866615 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603b2e8-5984-44ce-9e4c-5b6ad7d0fc6a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:27.961546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6e6cf-fb37-4c67-ac16-cec0eea56794 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:28.004567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc134bfd-654f-4ae9-b40e-afb1e69058b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:28.006077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc134bfd-654f-4ae9-b40e-afb1e69058b3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:28.006588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc134bfd-654f-4ae9-b40e-afb1e69058b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:28.007072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc134bfd-654f-4ae9-b40e-afb1e69058b3 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:28.008323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc134bfd-654f-4ae9-b40e-afb1e69058b3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:28.008848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc134bfd-654f-4ae9-b40e-afb1e69058b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:28.009099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603b2e8-5984-44ce-9e4c-5b6ad7d0fc6a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:28.010919 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc134bfd-654f-4ae9-b40e-afb1e69058b3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:28.049392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:28.051500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef92d5-496f-4878-8a86-2735e778c2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:28.052239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef92d5-496f-4878-8a86-2735e778c2fd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:28.052595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef92d5-496f-4878-8a86-2735e778c2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:28.052971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef92d5-496f-4878-8a86-2735e778c2fd None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:28.053863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef92d5-496f-4878-8a86-2735e778c2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:28.054142 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef92d5-496f-4878-8a86-2735e778c2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:28.055681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef92d5-496f-4878-8a86-2735e778c2fd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:28.063091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 260/526] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Jul 18 16:30:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:28.064259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc134bfd-654f-4ae9-b40e-afb1e69058b3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:28.069235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 267/527] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:28.070157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef92d5-496f-4878-8a86-2735e778c2fd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:28.145893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:28.178500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef92d5-496f-4878-8a86-2735e778c2fd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:28.178947 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc134bfd-654f-4ae9-b40e-afb1e69058b3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:28.214499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a736812-4b09-4b73-ba86-fde4fa4d2e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:28.215262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a736812-4b09-4b73-ba86-fde4fa4d2e00 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:28.215620 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a736812-4b09-4b73-ba86-fde4fa4d2e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:28.215958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b982a2-a106-4630-8d79-d49c9e67a36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:28.216611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a736812-4b09-4b73-ba86-fde4fa4d2e00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:28.217559 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b982a2-a106-4630-8d79-d49c9e67a36c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:28.217980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b982a2-a106-4630-8d79-d49c9e67a36c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:28.218417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b982a2-a106-4630-8d79-d49c9e67a36c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:28.237569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b982a2-a106-4630-8d79-d49c9e67a36c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:28.239285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b982a2-a106-4630-8d79-d49c9e67a36c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-82b982a2-a106-4630-8d79-d49c9e67a36c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:28.262546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:28.263757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:28.264819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:28.264819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:28.264819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:28.264819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:28.266616 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 261/528] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:28.267572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b982a2-a106-4630-8d79-d49c9e67a36c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:28.279512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:28.293462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a736812-4b09-4b73-ba86-fde4fa4d2e00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:28.295717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a736812-4b09-4b73-ba86-fde4fa4d2e00 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:28.383143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b982a2-a106-4630-8d79-d49c9e67a36c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:28.403335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 268/529] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:28.408688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a736812-4b09-4b73-ba86-fde4fa4d2e00 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:28.413488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:28.436143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ba725-4a61-49fb-ab53-780577ae12c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:28.437368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ba725-4a61-49fb-ab53-780577ae12c9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:28.437863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ba725-4a61-49fb-ab53-780577ae12c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:28.438427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ba725-4a61-49fb-ab53-780577ae12c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:28.447253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ba725-4a61-49fb-ab53-780577ae12c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:28.449554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ba725-4a61-49fb-ab53-780577ae12c9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-756ba725-4a61-49fb-ab53-780577ae12c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:28.467213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:28.468099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:28.468945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:28.468945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:28.468945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:28.468945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:28.468945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:28.468945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:28.468945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:28.468945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:28.471237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 262/530] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:28.472277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ba725-4a61-49fb-ab53-780577ae12c9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:28.502731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:28.518696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a736812-4b09-4b73-ba86-fde4fa4d2e00 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:28.557784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a0073-1ec3-4b6a-a199-23b79881a4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:28.558512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a0073-1ec3-4b6a-a199-23b79881a4bb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:28.558949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a0073-1ec3-4b6a-a199-23b79881a4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:28.559226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a0073-1ec3-4b6a-a199-23b79881a4bb None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:28.560538 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a0073-1ec3-4b6a-a199-23b79881a4bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:28.560844 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a0073-1ec3-4b6a-a199-23b79881a4bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:28.562436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a0073-1ec3-4b6a-a199-23b79881a4bb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:28.584156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 269/531] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:28.585838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a0073-1ec3-4b6a-a199-23b79881a4bb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:28.594169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ba725-4a61-49fb-ab53-780577ae12c9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:28.644996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d179b3-99e1-4381-8d05-d621ab66da3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:28.646601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d179b3-99e1-4381-8d05-d621ab66da3e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:28.647379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d179b3-99e1-4381-8d05-d621ab66da3e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:28.647870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d179b3-99e1-4381-8d05-d621ab66da3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:28.663060 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d179b3-99e1-4381-8d05-d621ab66da3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:28.665909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d179b3-99e1-4381-8d05-d621ab66da3e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c6d179b3-99e1-4381-8d05-d621ab66da3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:28.692963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:28.693935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:28.694805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:28.694805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:28.694805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:28.694805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:28.698766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 263/532] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:28.699612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d179b3-99e1-4381-8d05-d621ab66da3e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:28.727213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a0073-1ec3-4b6a-a199-23b79881a4bb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:28.739747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:28.774285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a381ec8-f2ff-45fe-a69b-4c182a58e7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:28.775911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a381ec8-f2ff-45fe-a69b-4c182a58e7df None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:28.776695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a381ec8-f2ff-45fe-a69b-4c182a58e7df None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:28.777217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a381ec8-f2ff-45fe-a69b-4c182a58e7df None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:28.779063 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a381ec8-f2ff-45fe-a69b-4c182a58e7df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:28.779736 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a381ec8-f2ff-45fe-a69b-4c182a58e7df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:28.781798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a381ec8-f2ff-45fe-a69b-4c182a58e7df None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:28.800993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 270/533] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:28.802664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a381ec8-f2ff-45fe-a69b-4c182a58e7df None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:28.813208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:28.838483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d179b3-99e1-4381-8d05-d621ab66da3e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:28.878342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347eb27-5205-4c60-9ede-2721d1455dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:28.879145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347eb27-5205-4c60-9ede-2721d1455dca None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:28.879408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347eb27-5205-4c60-9ede-2721d1455dca None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:28.879698 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347eb27-5205-4c60-9ede-2721d1455dca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:28.892956 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347eb27-5205-4c60-9ede-2721d1455dca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:28.895107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347eb27-5205-4c60-9ede-2721d1455dca None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-7347eb27-5205-4c60-9ede-2721d1455dca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:28.910232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:28.911124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:28.911957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:28.911957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:28.911957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:28.911957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:28.911957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:28.911957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:28.911957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:28.911957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:28.911957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 264/534] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:28.913043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347eb27-5205-4c60-9ede-2721d1455dca None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:28.948908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a381ec8-f2ff-45fe-a69b-4c182a58e7df None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:28.994986 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfc693-c80c-45f3-b87f-a8f2599d2326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:28.996022 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfc693-c80c-45f3-b87f-a8f2599d2326 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:28.996526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfc693-c80c-45f3-b87f-a8f2599d2326 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:28.997037 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfc693-c80c-45f3-b87f-a8f2599d2326 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:28.998005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfc693-c80c-45f3-b87f-a8f2599d2326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:28.998535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfc693-c80c-45f3-b87f-a8f2599d2326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:29.000178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfc693-c80c-45f3-b87f-a8f2599d2326 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:29.015963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 271/535] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:29.017222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfc693-c80c-45f3-b87f-a8f2599d2326 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:29.025975 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:29.037132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347eb27-5205-4c60-9ede-2721d1455dca None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:29.077208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea3c94-a3cd-44d7-a17b-9a3269420d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:29.077872 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea3c94-a3cd-44d7-a17b-9a3269420d60 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:29.078154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea3c94-a3cd-44d7-a17b-9a3269420d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:29.078430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea3c94-a3cd-44d7-a17b-9a3269420d60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:29.079270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea3c94-a3cd-44d7-a17b-9a3269420d60 None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=54c7869983bf4534b3e730af0185eac7) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:29.079575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea3c94-a3cd-44d7-a17b-9a3269420d60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:29.081173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea3c94-a3cd-44d7-a17b-9a3269420d60 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:29.087836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 265/536] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:28 2022] GET /identity/v3/role_assignments?user.id=54c7869983bf4534b3e730af0185eac7&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:29.097479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:29.099351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea3c94-a3cd-44d7-a17b-9a3269420d60 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:29.138291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfc693-c80c-45f3-b87f-a8f2599d2326 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:29.177037 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb99f8-e6a0-4a55-8572-fe88b9516ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:29.178099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb99f8-e6a0-4a55-8572-fe88b9516ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:29.178638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb99f8-e6a0-4a55-8572-fe88b9516ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:29.179153 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb99f8-e6a0-4a55-8572-fe88b9516ba3 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:29.180175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb99f8-e6a0-4a55-8572-fe88b9516ba3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:29.180691 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb99f8-e6a0-4a55-8572-fe88b9516ba3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:29.182402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb99f8-e6a0-4a55-8572-fe88b9516ba3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:29.196896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 272/537] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:29.206446 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb99f8-e6a0-4a55-8572-fe88b9516ba3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:29.280466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea3c94-a3cd-44d7-a17b-9a3269420d60 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:29.318174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d4de6-ea12-48ef-8ef0-33b2c676e20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:29.319018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d4de6-ea12-48ef-8ef0-33b2c676e20f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:29.319308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d4de6-ea12-48ef-8ef0-33b2c676e20f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:29.319593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d4de6-ea12-48ef-8ef0-33b2c676e20f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:29.329582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d4de6-ea12-48ef-8ef0-33b2c676e20f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:29.331066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d4de6-ea12-48ef-8ef0-33b2c676e20f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-910d4de6-ea12-48ef-8ef0-33b2c676e20f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:29.352907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:29.353823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:29.354639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:29.354639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:29.354639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:29.354639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:29.354639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:29.354639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:29.354639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:29.354639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:29.358687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 266/538] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:29.359896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d4de6-ea12-48ef-8ef0-33b2c676e20f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:29.381264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:29.425577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb99f8-e6a0-4a55-8572-fe88b9516ba3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:29.455765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:29.467523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca341f6-01be-4bd0-aa51-c6f64b705713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:29.468543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca341f6-01be-4bd0-aa51-c6f64b705713 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:29.469080 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca341f6-01be-4bd0-aa51-c6f64b705713 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:29.471474 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca341f6-01be-4bd0-aa51-c6f64b705713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:29.472614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca341f6-01be-4bd0-aa51-c6f64b705713 None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=de8484c3b114467ea59bebb2460ed546) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:29.473617 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca341f6-01be-4bd0-aa51-c6f64b705713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:29.475347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca341f6-01be-4bd0-aa51-c6f64b705713 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:29.483301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 273/539] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:29 2022] GET /identity/v3/role_assignments?user.id=de8484c3b114467ea59bebb2460ed546&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:29.564499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4dbca-468c-45d7-99ad-92ee581dc984 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:29.564845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4dbca-468c-45d7-99ad-92ee581dc984 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:29.565112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4dbca-468c-45d7-99ad-92ee581dc984 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:29.566796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 274/540] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:30:29.572474 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d1ee7-bbf1-4e64-b624-5630e6f25ba4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:29.572769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d1ee7-bbf1-4e64-b624-5630e6f25ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:29.573095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d1ee7-bbf1-4e64-b624-5630e6f25ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:29.599657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d4de6-ea12-48ef-8ef0-33b2c676e20f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:29.602717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-775d1ee7-bbf1-4e64-b624-5630e6f25ba4 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:29.603969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-775d1ee7-bbf1-4e64-b624-5630e6f25ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:29.668875 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b188e0f-70d9-4d3c-bf7a-092ff8cde976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:29.670199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188e0f-70d9-4d3c-bf7a-092ff8cde976 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:29.670481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188e0f-70d9-4d3c-bf7a-092ff8cde976 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:29.670754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188e0f-70d9-4d3c-bf7a-092ff8cde976 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:29.671561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b188e0f-70d9-4d3c-bf7a-092ff8cde976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:29.671866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b188e0f-70d9-4d3c-bf7a-092ff8cde976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:29.673367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b188e0f-70d9-4d3c-bf7a-092ff8cde976 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:29.716429 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 267/541] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:29.723224 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b188e0f-70d9-4d3c-bf7a-092ff8cde976 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:29.740276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 275/542] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:30:29.782970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d1ee7-bbf1-4e64-b624-5630e6f25ba4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:29.914091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:29.930346 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d1ee7-bbf1-4e64-b624-5630e6f25ba4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:29.971143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b188e0f-70d9-4d3c-bf7a-092ff8cde976 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:29.973931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:30.001618 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e18b1-f208-4c30-8ba3-93c94f41eecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:30.002870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e18b1-f208-4c30-8ba3-93c94f41eecb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:30.004530 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e18b1-f208-4c30-8ba3-93c94f41eecb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:30.005013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e18b1-f208-4c30-8ba3-93c94f41eecb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:30.025901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e18b1-f208-4c30-8ba3-93c94f41eecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-b74e18b1-f208-4c30-8ba3-93c94f41eecb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:30.035027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:30.035931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:30.036722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 276/543] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:30.042166 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74133d8-0c22-494b-a4d9-6d62a01ec82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:30.042337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61070fd2-e088-4f4d-b184-e827cc7c2654 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:30.042557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61070fd2-e088-4f4d-b184-e827cc7c2654 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:30.042847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61070fd2-e088-4f4d-b184-e827cc7c2654 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:30.043628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74133d8-0c22-494b-a4d9-6d62a01ec82e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:30.044661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74133d8-0c22-494b-a4d9-6d62a01ec82e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:30.048665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 277/544] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:30.048665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61070fd2-e088-4f4d-b184-e827cc7c2654 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:30.049396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74133d8-0c22-494b-a4d9-6d62a01ec82e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:30.050300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74133d8-0c22-494b-a4d9-6d62a01ec82e None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=809604baaed7486fb1020263e99e9c63) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:30.050619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74133d8-0c22-494b-a4d9-6d62a01ec82e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:30.052099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74133d8-0c22-494b-a4d9-6d62a01ec82e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:30.060468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 268/545] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:29 2022] GET /identity/v3/role_assignments?user.id=809604baaed7486fb1020263e99e9c63&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 259 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:30.061794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ed76-fb90-4154-b592-a2b35a3e24fa None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:30.062085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ed76-fb90-4154-b592-a2b35a3e24fa None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:30.062360 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ed76-fb90-4154-b592-a2b35a3e24fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:30.090542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-af14ed76-fb90-4154-b592-a2b35a3e24fa None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:30.091968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-af14ed76-fb90-4154-b592-a2b35a3e24fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:30.224044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 269/546] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:30.248798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14ed76-fb90-4154-b592-a2b35a3e24fa None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:30.265130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61070fd2-e088-4f4d-b184-e827cc7c2654 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:30.304468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c45d2a-a32f-4a48-9931-f707bfc5a353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:30.305314 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c45d2a-a32f-4a48-9931-f707bfc5a353 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:30.305607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c45d2a-a32f-4a48-9931-f707bfc5a353 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:30.305884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c45d2a-a32f-4a48-9931-f707bfc5a353 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:30.306713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c45d2a-a32f-4a48-9931-f707bfc5a353 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:30.307023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c45d2a-a32f-4a48-9931-f707bfc5a353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:30.308529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c45d2a-a32f-4a48-9931-f707bfc5a353 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:30.318577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 278/547] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:30.323758 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c45d2a-a32f-4a48-9931-f707bfc5a353 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:30.376517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:30.442342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c45d2a-a32f-4a48-9931-f707bfc5a353 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:30.478919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf0c6c-ec98-486b-bd0e-54651a5de644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:30.480811 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0c6c-ec98-486b-bd0e-54651a5de644 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:30.480811 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0c6c-ec98-486b-bd0e-54651a5de644 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:30.481135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf0c6c-ec98-486b-bd0e-54651a5de644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:30.497967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf0c6c-ec98-486b-bd0e-54651a5de644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:30.499523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf0c6c-ec98-486b-bd0e-54651a5de644 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f4cf0c6c-ec98-486b-bd0e-54651a5de644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:30.518393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:30.519278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:30.520610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:30.520610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:30.520610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:30.520610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:30.520610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 279/548] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:30.525891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf0c6c-ec98-486b-bd0e-54651a5de644 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:30.561281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:30.624645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14ed76-fb90-4154-b592-a2b35a3e24fa None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:30.682599 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:30.696187 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf0c6c-ec98-486b-bd0e-54651a5de644 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:30.707626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aefc9-903c-43a0-bd01-a3117a148385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:30.707626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aefc9-903c-43a0-bd01-a3117a148385 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:30.707626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aefc9-903c-43a0-bd01-a3117a148385 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:30.707626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90aefc9-903c-43a0-bd01-a3117a148385 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:30.733704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90aefc9-903c-43a0-bd01-a3117a148385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a90aefc9-903c-43a0-bd01-a3117a148385 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:30.741413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:30.742295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:30.743374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:30.748915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 270/549] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 495 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:30.751324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aefc9-903c-43a0-bd01-a3117a148385 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:30.760577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b0d99-25fd-497c-a57a-99eb09a93089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:30.761331 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b0d99-25fd-497c-a57a-99eb09a93089 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:30.761625 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b0d99-25fd-497c-a57a-99eb09a93089 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:30.761900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b0d99-25fd-497c-a57a-99eb09a93089 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:30.762635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b0d99-25fd-497c-a57a-99eb09a93089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:30.763516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b0d99-25fd-497c-a57a-99eb09a93089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:30.764914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b0d99-25fd-497c-a57a-99eb09a93089 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:30.801443 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 280/550] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:30.802430 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b0d99-25fd-497c-a57a-99eb09a93089 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:30.955403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90aefc9-903c-43a0-bd01-a3117a148385 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:31.007188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:31.024038 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ee296-3e2b-4082-b188-d636bbeb4927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:31.024720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ee296-3e2b-4082-b188-d636bbeb4927 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:31.025828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ee296-3e2b-4082-b188-d636bbeb4927 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:31.026114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ee296-3e2b-4082-b188-d636bbeb4927 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:31.026877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ee296-3e2b-4082-b188-d636bbeb4927 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:31.027186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ee296-3e2b-4082-b188-d636bbeb4927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:31.028713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ee296-3e2b-4082-b188-d636bbeb4927 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:31.045225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b0d99-25fd-497c-a57a-99eb09a93089 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:31.049622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 271/551] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:31.055431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ee296-3e2b-4082-b188-d636bbeb4927 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:31.114546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3673b-495f-419a-a947-94b49914a52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:31.115354 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3673b-495f-419a-a947-94b49914a52e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:31.115635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3673b-495f-419a-a947-94b49914a52e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:31.115910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3673b-495f-419a-a947-94b49914a52e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:31.133858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3673b-495f-419a-a947-94b49914a52e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:31.135330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3673b-495f-419a-a947-94b49914a52e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-30b3673b-495f-419a-a947-94b49914a52e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:31.171338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:31.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:31.173345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:31.173345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:31.173345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:31.173345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:31.173345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:31.173345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:31.173345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:31.173345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:31.173345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:30:31.173345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:31.180557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 281/552] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:31.188338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3673b-495f-419a-a947-94b49914a52e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:31.229239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:31.257781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ee296-3e2b-4082-b188-d636bbeb4927 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:31.332871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b0c8b-d17d-4077-8c0d-c79a47c4384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:31.333738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b0c8b-d17d-4077-8c0d-c79a47c4384e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:31.334019 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b0c8b-d17d-4077-8c0d-c79a47c4384e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:31.334308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b0c8b-d17d-4077-8c0d-c79a47c4384e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:31.358038 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b0c8b-d17d-4077-8c0d-c79a47c4384e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:31.359527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b0c8b-d17d-4077-8c0d-c79a47c4384e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-117b0c8b-d17d-4077-8c0d-c79a47c4384e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:31.394394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:31.395343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:31.396191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:31.396191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:31.396191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:31.396191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:31.397650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3673b-495f-419a-a947-94b49914a52e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:31.400410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 272/553] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:31.407430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b0c8b-d17d-4077-8c0d-c79a47c4384e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:31.447994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed8a63-3ae4-4b66-9fd4-f8ec7c4dafad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:31.448673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed8a63-3ae4-4b66-9fd4-f8ec7c4dafad None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:31.449007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed8a63-3ae4-4b66-9fd4-f8ec7c4dafad None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:31.449293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed8a63-3ae4-4b66-9fd4-f8ec7c4dafad None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:31.456905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed8a63-3ae4-4b66-9fd4-f8ec7c4dafad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:31.456905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed8a63-3ae4-4b66-9fd4-f8ec7c4dafad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:31.456905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed8a63-3ae4-4b66-9fd4-f8ec7c4dafad None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:31.473429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 282/554] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Jul 18 16:30:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:31.494311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed8a63-3ae4-4b66-9fd4-f8ec7c4dafad None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:31.576982 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:31.648911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:31.688704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b0c8b-d17d-4077-8c0d-c79a47c4384e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:31.689382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed8a63-3ae4-4b66-9fd4-f8ec7c4dafad None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:31.741674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65758b-0aa6-49ab-aa51-e1eba6fce2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:31.744914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65758b-0aa6-49ab-aa51-e1eba6fce2ce None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:31.744914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65758b-0aa6-49ab-aa51-e1eba6fce2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:31.744914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65758b-0aa6-49ab-aa51-e1eba6fce2ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:31.751364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b3f32-60d0-4d3a-b194-b4125753a5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:31.756906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b3f32-60d0-4d3a-b194-b4125753a5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:31.756906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b3f32-60d0-4d3a-b194-b4125753a5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:31.756906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b3f32-60d0-4d3a-b194-b4125753a5b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:31.756906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b3f32-60d0-4d3a-b194-b4125753a5b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:31.756906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b3f32-60d0-4d3a-b194-b4125753a5b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:31.756906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b3f32-60d0-4d3a-b194-b4125753a5b8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:31.796913 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 273/555] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:31.800903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ad06f-571c-465e-bce0-fdf2b21df63e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:31.800903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ad06f-571c-465e-bce0-fdf2b21df63e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:31.800903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ad06f-571c-465e-bce0-fdf2b21df63e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:31.812909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 274/556] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:31 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:31.812909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ad06f-571c-465e-bce0-fdf2b21df63e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:31.855939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65758b-0aa6-49ab-aa51-e1eba6fce2ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:31.859374 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65758b-0aa6-49ab-aa51-e1eba6fce2ce None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:31.966897 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:31.989907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ad06f-571c-465e-bce0-fdf2b21df63e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:31.992229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:32.010874 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 283/557] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 16:30:32.010874 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944e96f-e54a-45b4-8000-075ec66d6714 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.010874 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944e96f-e54a-45b4-8000-075ec66d6714 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.010874 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944e96f-e54a-45b4-8000-075ec66d6714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.032251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6944e96f-e54a-45b4-8000-075ec66d6714 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:32.033609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6944e96f-e54a-45b4-8000-075ec66d6714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:32.038677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27f0fb-8020-4e59-acf4-5d17ef6c7ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:32.039671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27f0fb-8020-4e59-acf4-5d17ef6c7ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.040168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27f0fb-8020-4e59-acf4-5d17ef6c7ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.040629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27f0fb-8020-4e59-acf4-5d17ef6c7ce4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.049650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27f0fb-8020-4e59-acf4-5d17ef6c7ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:32.051096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27f0fb-8020-4e59-acf4-5d17ef6c7ce4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-bf27f0fb-8020-4e59-acf4-5d17ef6c7ce4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:32.066031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:32.067568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:32.068493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:32.068493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:32.068493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:32.068493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:32.068493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:32.068493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:32.068493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:32.068493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:32.068493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:30:32.068493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:32.068493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 275/558] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Jul 18 16:30:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:32.069715 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71a1f3-993d-4520-8049-eb8ce62a5ea6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.069715 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71a1f3-993d-4520-8049-eb8ce62a5ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.069715 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71a1f3-993d-4520-8049-eb8ce62a5ea6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.072933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 276/559] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:32.072933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71a1f3-993d-4520-8049-eb8ce62a5ea6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:32.115326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 284/560] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:32.116403 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944e96f-e54a-45b4-8000-075ec66d6714 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:32.203439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71a1f3-993d-4520-8049-eb8ce62a5ea6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:32.234678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944e96f-e54a-45b4-8000-075ec66d6714 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:32.236233 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:32.242146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea06abe-862f-4e41-b1fa-f71fdf33e2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:32.246287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea06abe-862f-4e41-b1fa-f71fdf33e2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.246287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea06abe-862f-4e41-b1fa-f71fdf33e2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.246287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea06abe-862f-4e41-b1fa-f71fdf33e2d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.254451 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea06abe-862f-4e41-b1fa-f71fdf33e2d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:32.255986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea06abe-862f-4e41-b1fa-f71fdf33e2d4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-aea06abe-862f-4e41-b1fa-f71fdf33e2d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:32.277529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:32.279115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:32.284707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:32.284707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:32.284707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:32.284707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:32.284974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda2ec0-c95e-4465-bfd3-d241e2afe2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:32.284974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda2ec0-c95e-4465-bfd3-d241e2afe2ad None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.284974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda2ec0-c95e-4465-bfd3-d241e2afe2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.284974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda2ec0-c95e-4465-bfd3-d241e2afe2ad None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.285282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 277/561] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:32.285358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda2ec0-c95e-4465-bfd3-d241e2afe2ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:32.285358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda2ec0-c95e-4465-bfd3-d241e2afe2ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:32.285491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0621bb2-26cd-435b-8277-d8b7bde94ae1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.285491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0621bb2-26cd-435b-8277-d8b7bde94ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.285491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0621bb2-26cd-435b-8277-d8b7bde94ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.285689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda2ec0-c95e-4465-bfd3-d241e2afe2ad None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:32.304725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a0621bb2-26cd-435b-8277-d8b7bde94ae1 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:32.306083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a0621bb2-26cd-435b-8277-d8b7bde94ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:32.308306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 285/562] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Jul 18 16:30:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:32.309348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda2ec0-c95e-4465-bfd3-d241e2afe2ad None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:32.319780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:32.382318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 278/563] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:32.383408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0621bb2-26cd-435b-8277-d8b7bde94ae1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:32.420558 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda2ec0-c95e-4465-bfd3-d241e2afe2ad None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:32.455700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167af57-ab6e-41d8-8956-8d15943b9ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:32.456500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167af57-ab6e-41d8-8956-8d15943b9ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.456790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167af57-ab6e-41d8-8956-8d15943b9ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.457101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167af57-ab6e-41d8-8956-8d15943b9ef5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.467984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167af57-ab6e-41d8-8956-8d15943b9ef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e167af57-ab6e-41d8-8956-8d15943b9ef5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:32.475686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:32.476595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:32.477463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 286/564] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:32.478436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88ccfb-49bc-4411-9f55-f7e9c7ff19a5 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.478436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88ccfb-49bc-4411-9f55-f7e9c7ff19a5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.478769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88ccfb-49bc-4411-9f55-f7e9c7ff19a5 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.479916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 287/565] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:32.480601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88ccfb-49bc-4411-9f55-f7e9c7ff19a5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:32.493964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0621bb2-26cd-435b-8277-d8b7bde94ae1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:32.504290 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:32.529500 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965537d-fe80-491c-94f4-fedc1c948805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:32.530163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965537d-fe80-491c-94f4-fedc1c948805 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.530448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965537d-fe80-491c-94f4-fedc1c948805 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.530722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965537d-fe80-491c-94f4-fedc1c948805 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.531442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965537d-fe80-491c-94f4-fedc1c948805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:32.531806 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965537d-fe80-491c-94f4-fedc1c948805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:32.533230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965537d-fe80-491c-94f4-fedc1c948805 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:32.544652 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 279/566] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:32.545493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965537d-fe80-491c-94f4-fedc1c948805 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:32.550315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:32.587533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88ccfb-49bc-4411-9f55-f7e9c7ff19a5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:32.623536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef674c0-54bf-40b6-a8c8-1115531f6e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:32.624327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef674c0-54bf-40b6-a8c8-1115531f6e9a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.624618 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef674c0-54bf-40b6-a8c8-1115531f6e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.624917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef674c0-54bf-40b6-a8c8-1115531f6e9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.635693 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef674c0-54bf-40b6-a8c8-1115531f6e9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:32.637166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef674c0-54bf-40b6-a8c8-1115531f6e9a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:32.647588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965537d-fe80-491c-94f4-fedc1c948805 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-fef674c0-54bf-40b6-a8c8-1115531f6e9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:32.654851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:32.656016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:32.656930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:32.656930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:32.656930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:32.656930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:32.656930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 288/567] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:32.656930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef674c0-54bf-40b6-a8c8-1115531f6e9a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:32.684692 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7420e-97c1-4389-86a7-fbfd3ed5fd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:32.685530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7420e-97c1-4389-86a7-fbfd3ed5fd1b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.685816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7420e-97c1-4389-86a7-fbfd3ed5fd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.686092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7420e-97c1-4389-86a7-fbfd3ed5fd1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.696296 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7420e-97c1-4389-86a7-fbfd3ed5fd1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-10b7420e-97c1-4389-86a7-fbfd3ed5fd1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:32.703978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:32.704927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:32.705783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 280/568] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:32.706490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2ea7b-874d-49bb-89dc-9ab260db2fa8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.706824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2ea7b-874d-49bb-89dc-9ab260db2fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.707111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2ea7b-874d-49bb-89dc-9ab260db2fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.725061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6dd2ea7b-874d-49bb-89dc-9ab260db2fa8 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:32.726282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6dd2ea7b-874d-49bb-89dc-9ab260db2fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:32.735367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:32.764994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef674c0-54bf-40b6-a8c8-1115531f6e9a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:32.787338 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:32.796371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 281/569] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:32.797214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2ea7b-874d-49bb-89dc-9ab260db2fa8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:32.801377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae24aa-0263-49a0-b915-6afcb57e119a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:32.802188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae24aa-0263-49a0-b915-6afcb57e119a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.802469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae24aa-0263-49a0-b915-6afcb57e119a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.802742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae24aa-0263-49a0-b915-6afcb57e119a None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.803495 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae24aa-0263-49a0-b915-6afcb57e119a None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:32.803796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae24aa-0263-49a0-b915-6afcb57e119a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:32.805186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae24aa-0263-49a0-b915-6afcb57e119a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:32.813779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 289/570] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:32.814599 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae24aa-0263-49a0-b915-6afcb57e119a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:32.917811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2ea7b-874d-49bb-89dc-9ab260db2fa8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:32.952966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae24aa-0263-49a0-b915-6afcb57e119a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:32.956262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd95cc2-0bc6-47ab-abb0-55634e3cb084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:32.957118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd95cc2-0bc6-47ab-abb0-55634e3cb084 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.957405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd95cc2-0bc6-47ab-abb0-55634e3cb084 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.957813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd95cc2-0bc6-47ab-abb0-55634e3cb084 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.964808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd95cc2-0bc6-47ab-abb0-55634e3cb084 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:32.966898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd95cc2-0bc6-47ab-abb0-55634e3cb084 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-bfd95cc2-0bc6-47ab-abb0-55634e3cb084 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:32.980755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:32.982414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:32.983323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:32.983323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:32.983323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:32.983323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:32.983323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:32.983323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:32.983323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:32.983323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:32.983323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 282/571] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:32.983818 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd95cc2-0bc6-47ab-abb0-55634e3cb084 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:32.996843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad49383-2d1a-406b-a149-df0f26e44522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:32.997543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad49383-2d1a-406b-a149-df0f26e44522 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:32.997825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad49383-2d1a-406b-a149-df0f26e44522 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:32.998098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad49383-2d1a-406b-a149-df0f26e44522 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:32.998855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad49383-2d1a-406b-a149-df0f26e44522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:32.999161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad49383-2d1a-406b-a149-df0f26e44522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:33.000537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad49383-2d1a-406b-a149-df0f26e44522 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:33.002236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:33.019914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 290/572] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:33.020671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad49383-2d1a-406b-a149-df0f26e44522 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:33.046330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:33.089964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd95cc2-0bc6-47ab-abb0-55634e3cb084 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:33.122695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad49383-2d1a-406b-a149-df0f26e44522 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:33.125470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a4fec-20b6-40a8-82fe-3a92537113e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:33.126142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a4fec-20b6-40a8-82fe-3a92537113e0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.126426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a4fec-20b6-40a8-82fe-3a92537113e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.126702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a4fec-20b6-40a8-82fe-3a92537113e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.127453 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a4fec-20b6-40a8-82fe-3a92537113e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:33.127761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a4fec-20b6-40a8-82fe-3a92537113e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:33.129158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a4fec-20b6-40a8-82fe-3a92537113e0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:33.137520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 283/573] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:33.138853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a4fec-20b6-40a8-82fe-3a92537113e0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:33.160260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf7e08-f9e2-4939-95be-eac143bb1843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:33.161093 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf7e08-f9e2-4939-95be-eac143bb1843 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.161380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf7e08-f9e2-4939-95be-eac143bb1843 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.161665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf7e08-f9e2-4939-95be-eac143bb1843 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.171966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf7e08-f9e2-4939-95be-eac143bb1843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bbbf7e08-f9e2-4939-95be-eac143bb1843 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:33.179613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:33.180733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 291/574] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:33.181763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf7e08-f9e2-4939-95be-eac143bb1843 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:33.237313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:33.244407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a4fec-20b6-40a8-82fe-3a92537113e0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:33.262124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:33.281679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86491d40-5d94-41b8-823b-f58a7aeb268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:33.282478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86491d40-5d94-41b8-823b-f58a7aeb268f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.282764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86491d40-5d94-41b8-823b-f58a7aeb268f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.282896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf7e08-f9e2-4939-95be-eac143bb1843 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:33.283041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86491d40-5d94-41b8-823b-f58a7aeb268f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.294165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86491d40-5d94-41b8-823b-f58a7aeb268f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:33.295616 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86491d40-5d94-41b8-823b-f58a7aeb268f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-86491d40-5d94-41b8-823b-f58a7aeb268f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:33.313538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:33.314795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:33.316324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:33.316324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:33.316324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:33.316324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:33.316324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 284/575] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:33.316324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86491d40-5d94-41b8-823b-f58a7aeb268f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:33.337816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b1339-aa70-4e0d-a641-cbdd6b98289f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:33.338478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b1339-aa70-4e0d-a641-cbdd6b98289f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.338757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b1339-aa70-4e0d-a641-cbdd6b98289f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.339034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b1339-aa70-4e0d-a641-cbdd6b98289f None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.339826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b1339-aa70-4e0d-a641-cbdd6b98289f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:33.340138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b1339-aa70-4e0d-a641-cbdd6b98289f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:33.342212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b1339-aa70-4e0d-a641-cbdd6b98289f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:33.354315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 292/576] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:33.355339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b1339-aa70-4e0d-a641-cbdd6b98289f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:33.421710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86491d40-5d94-41b8-823b-f58a7aeb268f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:33.457431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7d417-fd16-485a-8037-5b10e0e6f873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:33.458142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7d417-fd16-485a-8037-5b10e0e6f873 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.458425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7d417-fd16-485a-8037-5b10e0e6f873 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.458694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7d417-fd16-485a-8037-5b10e0e6f873 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.461723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b1339-aa70-4e0d-a641-cbdd6b98289f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:33.465573 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7d417-fd16-485a-8037-5b10e0e6f873 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:33.467044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7d417-fd16-485a-8037-5b10e0e6f873 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:33.477240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-49a7d417-fd16-485a-8037-5b10e0e6f873 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:33.481385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:33.482264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:33.483145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:33.483145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:33.483145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:33.483145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:33.483145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:33.483145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:33.483145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:33.483145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:33.483145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 285/577] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:33.483643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7d417-fd16-485a-8037-5b10e0e6f873 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:33.503391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8c8dc-75aa-4c25-8b3d-9e9a55c59ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:33.504187 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8c8dc-75aa-4c25-8b3d-9e9a55c59ccf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.504473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8c8dc-75aa-4c25-8b3d-9e9a55c59ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.504803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8c8dc-75aa-4c25-8b3d-9e9a55c59ccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.511714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:33.516560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8c8dc-75aa-4c25-8b3d-9e9a55c59ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:33.518049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8c8dc-75aa-4c25-8b3d-9e9a55c59ccf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1ac8c8dc-75aa-4c25-8b3d-9e9a55c59ccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:33.535728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:33.536666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:33.537610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:33.537610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:33.537610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:33.537610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:33.537610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 293/578] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:33.537892 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8c8dc-75aa-4c25-8b3d-9e9a55c59ccf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:33.587562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7d417-fd16-485a-8037-5b10e0e6f873 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:33.621744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6eac1-0c5f-4faf-b3d6-2b7301e53e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:33.622905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6eac1-0c5f-4faf-b3d6-2b7301e53e3d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.623185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6eac1-0c5f-4faf-b3d6-2b7301e53e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.623461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6eac1-0c5f-4faf-b3d6-2b7301e53e3d None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.624198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6eac1-0c5f-4faf-b3d6-2b7301e53e3d None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:33.624498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6eac1-0c5f-4faf-b3d6-2b7301e53e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:33.625913 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6eac1-0c5f-4faf-b3d6-2b7301e53e3d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:33.634113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 286/579] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:33.634897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6eac1-0c5f-4faf-b3d6-2b7301e53e3d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:33.649016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8c8dc-75aa-4c25-8b3d-9e9a55c59ccf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:33.685616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764579c-4d3f-4325-9b69-e617f1c9e759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:33.686285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764579c-4d3f-4325-9b69-e617f1c9e759 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.686571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764579c-4d3f-4325-9b69-e617f1c9e759 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.686850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764579c-4d3f-4325-9b69-e617f1c9e759 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.687577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764579c-4d3f-4325-9b69-e617f1c9e759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:33.687886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764579c-4d3f-4325-9b69-e617f1c9e759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:33.689341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764579c-4d3f-4325-9b69-e617f1c9e759 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:33.701361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 294/580] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:33.702166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764579c-4d3f-4325-9b69-e617f1c9e759 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:33.729432 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:33.729596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:33.739668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6eac1-0c5f-4faf-b3d6-2b7301e53e3d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:33.788510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95361644-5494-4c95-99bd-c2c2ecb58989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:33.789423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95361644-5494-4c95-99bd-c2c2ecb58989 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.789706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95361644-5494-4c95-99bd-c2c2ecb58989 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.789983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95361644-5494-4c95-99bd-c2c2ecb58989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.790824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95361644-5494-4c95-99bd-c2c2ecb58989 None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=bf341f1bb8944a3e9331678100b941a1) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:33.791129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95361644-5494-4c95-99bd-c2c2ecb58989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:33.792677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95361644-5494-4c95-99bd-c2c2ecb58989 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:33.798902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 287/581] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:33 2022] GET /identity/v3/role_assignments?user.id=bf341f1bb8944a3e9331678100b941a1&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:33.799682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95361644-5494-4c95-99bd-c2c2ecb58989 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:33.814392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764579c-4d3f-4325-9b69-e617f1c9e759 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:33.858243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa500d49-1f95-4576-9bcd-7f883ed7de43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:33.858926 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa500d49-1f95-4576-9bcd-7f883ed7de43 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:33.859210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa500d49-1f95-4576-9bcd-7f883ed7de43 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:33.859488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa500d49-1f95-4576-9bcd-7f883ed7de43 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:33.860298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa500d49-1f95-4576-9bcd-7f883ed7de43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:33.860607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa500d49-1f95-4576-9bcd-7f883ed7de43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:33.862124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa500d49-1f95-4576-9bcd-7f883ed7de43 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:33.882407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 295/582] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:33.882407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa500d49-1f95-4576-9bcd-7f883ed7de43 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:33.961482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95361644-5494-4c95-99bd-c2c2ecb58989 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:33.989046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:33.999906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cdd0a-7e22-4781-bc40-0b993ccd9036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:34.000578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cdd0a-7e22-4781-bc40-0b993ccd9036 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:34.000868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cdd0a-7e22-4781-bc40-0b993ccd9036 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:34.001177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cdd0a-7e22-4781-bc40-0b993ccd9036 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:34.002450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cdd0a-7e22-4781-bc40-0b993ccd9036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:34.002450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cdd0a-7e22-4781-bc40-0b993ccd9036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:34.003607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cdd0a-7e22-4781-bc40-0b993ccd9036 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:34.011426 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa500d49-1f95-4576-9bcd-7f883ed7de43 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:34.021810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 288/583] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:34.021810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cdd0a-7e22-4781-bc40-0b993ccd9036 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:34.036463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:34.053026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bee2c5-b4dd-4c08-9e8d-ad405be3f42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:34.053988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bee2c5-b4dd-4c08-9e8d-ad405be3f42b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:34.054329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bee2c5-b4dd-4c08-9e8d-ad405be3f42b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:34.054614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bee2c5-b4dd-4c08-9e8d-ad405be3f42b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:34.066749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bee2c5-b4dd-4c08-9e8d-ad405be3f42b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:34.068369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bee2c5-b4dd-4c08-9e8d-ad405be3f42b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e4bee2c5-b4dd-4c08-9e8d-ad405be3f42b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:34.089382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:34.090320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:34.091228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:34.091228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:34.091228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:34.091228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:34.091228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 296/584] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:34.091524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bee2c5-b4dd-4c08-9e8d-ad405be3f42b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:34.133272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cdd0a-7e22-4781-bc40-0b993ccd9036 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:34.172261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e0d7b-d695-43b9-8429-9e53f46a6a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:34.173101 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e0d7b-d695-43b9-8429-9e53f46a6a55 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:34.173380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e0d7b-d695-43b9-8429-9e53f46a6a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:34.173650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e0d7b-d695-43b9-8429-9e53f46a6a55 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:34.181650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e0d7b-d695-43b9-8429-9e53f46a6a55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:34.183090 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e0d7b-d695-43b9-8429-9e53f46a6a55 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-893e0d7b-d695-43b9-8429-9e53f46a6a55 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:34.197710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:34.198702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 289/585] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:34.199651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e0d7b-d695-43b9-8429-9e53f46a6a55 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:34.219054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bee2c5-b4dd-4c08-9e8d-ad405be3f42b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:34.261456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e0d25-0cc3-4569-84a8-4377b3593f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:34.262360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e0d25-0cc3-4569-84a8-4377b3593f27 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:34.262636 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e0d25-0cc3-4569-84a8-4377b3593f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:34.262940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e0d25-0cc3-4569-84a8-4377b3593f27 None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/69ee145599384c6da2cba85a42832dcd/roles/3fc59073330c4557ad178629386b0852` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:34.271510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:34.279467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e0d25-0cc3-4569-84a8-4377b3593f27 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=69ee145599384c6da2cba85a42832dcd, role_id=3fc59073330c4557ad178629386b0852)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:34.280979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e0d25-0cc3-4569-84a8-4377b3593f27 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:34.289211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:34.314283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e0d7b-d695-43b9-8429-9e53f46a6a55 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:34.349157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662634a4-0e98-488c-864c-fba974e35c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:34.349965 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662634a4-0e98-488c-864c-fba974e35c97 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:34.350409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662634a4-0e98-488c-864c-fba974e35c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:34.350690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662634a4-0e98-488c-864c-fba974e35c97 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:34.359290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662634a4-0e98-488c-864c-fba974e35c97 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:34.360767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662634a4-0e98-488c-864c-fba974e35c97 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:34.363382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 297/586] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:30:34 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/69ee145599384c6da2cba85a42832dcd/roles/3fc59073330c4557ad178629386b0852 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:30:34.364138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e0d25-0cc3-4569-84a8-4377b3593f27 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-662634a4-0e98-488c-864c-fba974e35c97 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:34.390564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:34.391586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:34.392534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:34.392534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:34.392534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:34.392534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:34.392534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:34.392534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:34.392534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:34.392534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:34.392534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:30:34.392534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:34.392534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 290/587] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:34.393812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662634a4-0e98-488c-864c-fba974e35c97 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:34.522850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e0d25-0cc3-4569-84a8-4377b3593f27 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:34.549650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662634a4-0e98-488c-864c-fba974e35c97 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:34.578718 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b423092-fcbc-4fe3-838d-b4472c0bffa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:34.579439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b423092-fcbc-4fe3-838d-b4472c0bffa8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:34.579721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b423092-fcbc-4fe3-838d-b4472c0bffa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:34.579995 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b423092-fcbc-4fe3-838d-b4472c0bffa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:34.580767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b423092-fcbc-4fe3-838d-b4472c0bffa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:34.581233 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b423092-fcbc-4fe3-838d-b4472c0bffa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:34.582585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b423092-fcbc-4fe3-838d-b4472c0bffa8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:34.594966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 298/588] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:34.595826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b423092-fcbc-4fe3-838d-b4472c0bffa8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:34.596827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687a66c-7a71-4037-b601-ff8b3cbbf137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:34.597772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687a66c-7a71-4037-b601-ff8b3cbbf137 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:34.598283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687a66c-7a71-4037-b601-ff8b3cbbf137 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:34.598958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687a66c-7a71-4037-b601-ff8b3cbbf137 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:34.600987 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687a66c-7a71-4037-b601-ff8b3cbbf137 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:34.601523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687a66c-7a71-4037-b601-ff8b3cbbf137 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:34.603272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687a66c-7a71-4037-b601-ff8b3cbbf137 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:34.605933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:34.631018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 291/589] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Jul 18 16:30:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:34.632212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687a66c-7a71-4037-b601-ff8b3cbbf137 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:34.657230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:34.730088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b423092-fcbc-4fe3-838d-b4472c0bffa8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:34.761040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687a66c-7a71-4037-b601-ff8b3cbbf137 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:34.779038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df11657-93e3-4cd4-ab3b-167bcb2ebb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:34.780397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df11657-93e3-4cd4-ab3b-167bcb2ebb76 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:34.780908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df11657-93e3-4cd4-ab3b-167bcb2ebb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:34.781852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df11657-93e3-4cd4-ab3b-167bcb2ebb76 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:34.783174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df11657-93e3-4cd4-ab3b-167bcb2ebb76 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:34.783712 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df11657-93e3-4cd4-ab3b-167bcb2ebb76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:34.786098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df11657-93e3-4cd4-ab3b-167bcb2ebb76 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:34.809368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a4f59-957e-48a6-8b19-891d41b87cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:34.811136 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4f59-957e-48a6-8b19-891d41b87cdb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:34.811627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4f59-957e-48a6-8b19-891d41b87cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:34.812136 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4f59-957e-48a6-8b19-891d41b87cdb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:34.813253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 299/590] 10.222.0.88 () {56 vars in 1139 bytes} [Mon Jul 18 16:30:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:34.814026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df11657-93e3-4cd4-ab3b-167bcb2ebb76 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:34.822986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a4f59-957e-48a6-8b19-891d41b87cdb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:34.824784 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a4f59-957e-48a6-8b19-891d41b87cdb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-9c2a4f59-957e-48a6-8b19-891d41b87cdb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:34.846042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:34.847068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:34.848044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:34.848044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:34.848044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:34.848044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:34.848044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:34.848044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:34.848044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:34.848044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:34.848044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:30:34.848044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:34.850647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 292/591] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:34.851858 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a4f59-957e-48a6-8b19-891d41b87cdb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:34.921602 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:34.953520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df11657-93e3-4cd4-ab3b-167bcb2ebb76 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:34.963011 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:34.984783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a4f59-957e-48a6-8b19-891d41b87cdb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:35.001345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d2d29-7d16-4d0c-802a-1036aadf3456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:35.002503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d2d29-7d16-4d0c-802a-1036aadf3456 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:35.002993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d2d29-7d16-4d0c-802a-1036aadf3456 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:35.003501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d2d29-7d16-4d0c-802a-1036aadf3456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:35.015659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d2d29-7d16-4d0c-802a-1036aadf3456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:35.017865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d2d29-7d16-4d0c-802a-1036aadf3456 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:35.026181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287830c7-160b-4db4-a69e-bc2c4f0ec663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:35.026602 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287830c7-160b-4db4-a69e-bc2c4f0ec663 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:35.026884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287830c7-160b-4db4-a69e-bc2c4f0ec663 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:35.027154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287830c7-160b-4db4-a69e-bc2c4f0ec663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:35.038786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287830c7-160b-4db4-a69e-bc2c4f0ec663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4e9d2d29-7d16-4d0c-802a-1036aadf3456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:35.040132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:35.041055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:35.041956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:35.041956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:35.041956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:35.041956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:35.042186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287830c7-160b-4db4-a69e-bc2c4f0ec663 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:35.044610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 300/592] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:35.045861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d2d29-7d16-4d0c-802a-1036aadf3456 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-287830c7-160b-4db4-a69e-bc2c4f0ec663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:35.061218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:35.062155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:35.063070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:35.063070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:35.063070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:35.063070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:35.063070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 293/593] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:35.063763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287830c7-160b-4db4-a69e-bc2c4f0ec663 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:35.166326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d2d29-7d16-4d0c-802a-1036aadf3456 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:35.174235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287830c7-160b-4db4-a69e-bc2c4f0ec663 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:35.176662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:35.215009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d031d2-9c73-45b3-acc0-92ad2afd4a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:35.215223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e6124-716f-4566-9b83-996403ffa515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:35.220900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d031d2-9c73-45b3-acc0-92ad2afd4a17 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:35.221058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6124-716f-4566-9b83-996403ffa515 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:35.221143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d031d2-9c73-45b3-acc0-92ad2afd4a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:35.221216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6124-716f-4566-9b83-996403ffa515 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:35.221284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d031d2-9c73-45b3-acc0-92ad2afd4a17 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:35.221353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6124-716f-4566-9b83-996403ffa515 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:35.221430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d031d2-9c73-45b3-acc0-92ad2afd4a17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:35.221526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e6124-716f-4566-9b83-996403ffa515 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:35.221611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d031d2-9c73-45b3-acc0-92ad2afd4a17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:35.221693 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e6124-716f-4566-9b83-996403ffa515 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:35.221774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d031d2-9c73-45b3-acc0-92ad2afd4a17 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:35.221854 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e6124-716f-4566-9b83-996403ffa515 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:35.232636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 294/594] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:35.233518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d031d2-9c73-45b3-acc0-92ad2afd4a17 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:35.245207 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 301/595] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Jul 18 16:30:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:35.246132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e6124-716f-4566-9b83-996403ffa515 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:35.271504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:35.360511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d031d2-9c73-45b3-acc0-92ad2afd4a17 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:35.388229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e6124-716f-4566-9b83-996403ffa515 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:35.398498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da984e3-fa08-4c5e-b130-9ea3f39aa0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:35.399290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da984e3-fa08-4c5e-b130-9ea3f39aa0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:35.399642 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da984e3-fa08-4c5e-b130-9ea3f39aa0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:35.399994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da984e3-fa08-4c5e-b130-9ea3f39aa0d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:35.400808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da984e3-fa08-4c5e-b130-9ea3f39aa0d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:35.401213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da984e3-fa08-4c5e-b130-9ea3f39aa0d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:35.402683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da984e3-fa08-4c5e-b130-9ea3f39aa0d6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:35.424080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 295/596] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:35.425183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da984e3-fa08-4c5e-b130-9ea3f39aa0d6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:35.435358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e147786-0d6a-4c08-b079-ad69fe83ab76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:35.436726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e147786-0d6a-4c08-b079-ad69fe83ab76 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:35.440913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e147786-0d6a-4c08-b079-ad69fe83ab76 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:35.440913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e147786-0d6a-4c08-b079-ad69fe83ab76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:35.448907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e147786-0d6a-4c08-b079-ad69fe83ab76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:35.449095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e147786-0d6a-4c08-b079-ad69fe83ab76 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:35.455053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9e147786-0d6a-4c08-b079-ad69fe83ab76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:35.470158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:35.471103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:35.471973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:35.471973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:35.471973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:35.471973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:35.471973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:35.471973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:35.471973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:35.471973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:35.474490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 302/597] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:35.475512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e147786-0d6a-4c08-b079-ad69fe83ab76 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:35.554313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da984e3-fa08-4c5e-b130-9ea3f39aa0d6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:35.578654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:35.603492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e147786-0d6a-4c08-b079-ad69fe83ab76 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:35.605734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbedb4-c390-41b8-af5d-9049a1966552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:35.608175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbedb4-c390-41b8-af5d-9049a1966552 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:35.608609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbedb4-c390-41b8-af5d-9049a1966552 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:35.609072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbedb4-c390-41b8-af5d-9049a1966552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:35.623118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbedb4-c390-41b8-af5d-9049a1966552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:35.625256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dbedb4-c390-41b8-af5d-9049a1966552 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d2dbedb4-c390-41b8-af5d-9049a1966552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:35.647797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:35.649025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:35.649985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:35.649985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:35.649985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:35.649985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:35.651676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 296/598] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:35.652743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbedb4-c390-41b8-af5d-9049a1966552 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:35.658487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf2993-de18-4a0c-ba41-999129048b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:35.658980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf2993-de18-4a0c-ba41-999129048b65 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:35.659339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf2993-de18-4a0c-ba41-999129048b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:35.659691 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf2993-de18-4a0c-ba41-999129048b65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:35.667480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf2993-de18-4a0c-ba41-999129048b65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:35.668721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf2993-de18-4a0c-ba41-999129048b65 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1adf2993-de18-4a0c-ba41-999129048b65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:35.684260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:35.685286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:35.687624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:35.687624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:35.687624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:35.687624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:35.687624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:35.687624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:35.687624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:35.687624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:35.687624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 303/599] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:35.687624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf2993-de18-4a0c-ba41-999129048b65 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:35.757463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:35.785352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbedb4-c390-41b8-af5d-9049a1966552 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:35.811055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf2993-de18-4a0c-ba41-999129048b65 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:35.824547 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff702374-7960-4ff6-8389-2ddef1c0b3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:35.825317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff702374-7960-4ff6-8389-2ddef1c0b3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:35.825601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff702374-7960-4ff6-8389-2ddef1c0b3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:35.825883 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff702374-7960-4ff6-8389-2ddef1c0b3e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:35.826678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff702374-7960-4ff6-8389-2ddef1c0b3e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:35.826986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff702374-7960-4ff6-8389-2ddef1c0b3e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:35.828667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff702374-7960-4ff6-8389-2ddef1c0b3e2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:35.848054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 297/600] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:35.848988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff702374-7960-4ff6-8389-2ddef1c0b3e2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:35.853636 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e9f04-9eb3-4cb1-85ec-bd3050e3a287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:35.854317 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e9f04-9eb3-4cb1-85ec-bd3050e3a287 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:35.854598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e9f04-9eb3-4cb1-85ec-bd3050e3a287 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:35.855612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e9f04-9eb3-4cb1-85ec-bd3050e3a287 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:35.855612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e9f04-9eb3-4cb1-85ec-bd3050e3a287 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:35.855898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e9f04-9eb3-4cb1-85ec-bd3050e3a287 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:35.860970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e9f04-9eb3-4cb1-85ec-bd3050e3a287 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:35.875095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:35.882166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 304/601] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:35.883396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e9f04-9eb3-4cb1-85ec-bd3050e3a287 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:35.985268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff702374-7960-4ff6-8389-2ddef1c0b3e2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:36.013689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e9f04-9eb3-4cb1-85ec-bd3050e3a287 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:36.023156 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabece1-5508-486b-bc29-935993a3307c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:36.023924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabece1-5508-486b-bc29-935993a3307c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:36.024210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabece1-5508-486b-bc29-935993a3307c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:36.024493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabece1-5508-486b-bc29-935993a3307c None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:36.025329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabece1-5508-486b-bc29-935993a3307c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:36.025635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabece1-5508-486b-bc29-935993a3307c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:36.027091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabece1-5508-486b-bc29-935993a3307c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:36.038691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:36.052595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 298/602] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:36.057893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabece1-5508-486b-bc29-935993a3307c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:36.067135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999868f-c594-497a-83ce-6ba7742692d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:36.068047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999868f-c594-497a-83ce-6ba7742692d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:36.068334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999868f-c594-497a-83ce-6ba7742692d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:36.068612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999868f-c594-497a-83ce-6ba7742692d6 None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/54c7869983bf4534b3e730af0185eac7/roles/3fc59073330c4557ad178629386b0852` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:36.096132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999868f-c594-497a-83ce-6ba7742692d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=54c7869983bf4534b3e730af0185eac7, role_id=3fc59073330c4557ad178629386b0852)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:36.097202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999868f-c594-497a-83ce-6ba7742692d6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:36.180026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabece1-5508-486b-bc29-935993a3307c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:36.185755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 305/603] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:30:35 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/54c7869983bf4534b3e730af0185eac7/roles/3fc59073330c4557ad178629386b0852 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:30:36.194535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88d161-8be9-4602-90d0-7b637b2a7ece None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:36.194855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88d161-8be9-4602-90d0-7b637b2a7ece None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:36.201771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88d161-8be9-4602-90d0-7b637b2a7ece None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:36.201771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 306/604] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:36.201771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88d161-8be9-4602-90d0-7b637b2a7ece None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:36.234431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d00e31-06ab-49b0-a483-f6dcd5c66eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:36.238702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d00e31-06ab-49b0-a483-f6dcd5c66eec None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:36.238702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d00e31-06ab-49b0-a483-f6dcd5c66eec None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:36.238702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d00e31-06ab-49b0-a483-f6dcd5c66eec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:36.252998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d00e31-06ab-49b0-a483-f6dcd5c66eec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:36.252998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d00e31-06ab-49b0-a483-f6dcd5c66eec None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-e3d00e31-06ab-49b0-a483-f6dcd5c66eec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:36.272170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:36.273408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:36.274285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:36.274285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:36.274285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:36.274285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:36.274285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:36.274285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:36.274285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:36.274285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:36.274285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 299/605] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:36.274285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bacaae-b979-40e3-b3bb-f60661077012 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:36.274285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bacaae-b979-40e3-b3bb-f60661077012 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:36.274878 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bacaae-b979-40e3-b3bb-f60661077012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:36.282596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:36.306395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f7bacaae-b979-40e3-b3bb-f60661077012 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:36.307650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f7bacaae-b979-40e3-b3bb-f60661077012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:36.414015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:36.415362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88d161-8be9-4602-90d0-7b637b2a7ece None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:36.444986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 300/606] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:36.444986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bacaae-b979-40e3-b3bb-f60661077012 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:36.499855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb096a37-e2a5-414a-b988-5c1db8eb3809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:36.501250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb096a37-e2a5-414a-b988-5c1db8eb3809 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:36.501375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb096a37-e2a5-414a-b988-5c1db8eb3809 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:36.504903 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb096a37-e2a5-414a-b988-5c1db8eb3809 None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/de8484c3b114467ea59bebb2460ed546/roles/3fc59073330c4557ad178629386b0852` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:36.528913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb096a37-e2a5-414a-b988-5c1db8eb3809 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=de8484c3b114467ea59bebb2460ed546, role_id=3fc59073330c4557ad178629386b0852)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:36.528913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb096a37-e2a5-414a-b988-5c1db8eb3809 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:36.608522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 307/607] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:30:36 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/de8484c3b114467ea59bebb2460ed546/roles/3fc59073330c4557ad178629386b0852 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:30:36.609392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb096a37-e2a5-414a-b988-5c1db8eb3809 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:36.625684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bacaae-b979-40e3-b3bb-f60661077012 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:36.698633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764d2ec-8a99-422a-bb9f-8e51ddbe2d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:36.699678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764d2ec-8a99-422a-bb9f-8e51ddbe2d79 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:36.700268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764d2ec-8a99-422a-bb9f-8e51ddbe2d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:36.700754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764d2ec-8a99-422a-bb9f-8e51ddbe2d79 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:36.701823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764d2ec-8a99-422a-bb9f-8e51ddbe2d79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:36.702346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764d2ec-8a99-422a-bb9f-8e51ddbe2d79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:36.704026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764d2ec-8a99-422a-bb9f-8e51ddbe2d79 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:36.739051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 301/608] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:36.740944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98522bb-0cbc-43fa-8590-c9188c8edf15 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:36.741446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98522bb-0cbc-43fa-8590-c9188c8edf15 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:36.741922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98522bb-0cbc-43fa-8590-c9188c8edf15 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:36.746170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 302/609] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:36.748272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98522bb-0cbc-43fa-8590-c9188c8edf15 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:36.826728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:36.835798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb096a37-e2a5-414a-b988-5c1db8eb3809 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:36.858525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:36.914179 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772c472-c2db-4b40-b377-46032ae8b3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:36.915002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c472-c2db-4b40-b377-46032ae8b3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:36.915280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c472-c2db-4b40-b377-46032ae8b3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:36.915553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c472-c2db-4b40-b377-46032ae8b3c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:36.944907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772c472-c2db-4b40-b377-46032ae8b3c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-3772c472-c2db-4b40-b377-46032ae8b3c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:36.952972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:36.953988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 308/610] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:36.954827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31fb1f-080c-4710-b167-e25090d195d1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:36.955575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31fb1f-080c-4710-b167-e25090d195d1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:36.955575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31fb1f-080c-4710-b167-e25090d195d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:36.984983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98522bb-0cbc-43fa-8590-c9188c8edf15 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:37.004910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bc31fb1f-080c-4710-b167-e25090d195d1 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:37.004910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bc31fb1f-080c-4710-b167-e25090d195d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:37.077797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb7a03-eb88-45ff-88ca-8e7f69a159fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:37.078828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb7a03-eb88-45ff-88ca-8e7f69a159fd None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:37.079278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb7a03-eb88-45ff-88ca-8e7f69a159fd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:37.080531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb7a03-eb88-45ff-88ca-8e7f69a159fd None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/809604baaed7486fb1020263e99e9c63/roles/3fc59073330c4557ad178629386b0852` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:37.112906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb7a03-eb88-45ff-88ca-8e7f69a159fd None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=809604baaed7486fb1020263e99e9c63, role_id=3fc59073330c4557ad178629386b0852)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:37.114456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb7a03-eb88-45ff-88ca-8e7f69a159fd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:37.115044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 309/611] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:37.115959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31fb1f-080c-4710-b167-e25090d195d1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:37.214959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:37.225398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 303/612] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:30:36 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/809604baaed7486fb1020263e99e9c63/roles/3fc59073330c4557ad178629386b0852 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:30:37.226383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb7a03-eb88-45ff-88ca-8e7f69a159fd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:37.273829 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31fb1f-080c-4710-b167-e25090d195d1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:37.362818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901b00d-e2af-4a3e-9bb5-5089f1e9d6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:37.364917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901b00d-e2af-4a3e-9bb5-5089f1e9d6bf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:37.364917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901b00d-e2af-4a3e-9bb5-5089f1e9d6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:37.364917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901b00d-e2af-4a3e-9bb5-5089f1e9d6bf None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:37.364917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901b00d-e2af-4a3e-9bb5-5089f1e9d6bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:37.365298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901b00d-e2af-4a3e-9bb5-5089f1e9d6bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:37.366517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901b00d-e2af-4a3e-9bb5-5089f1e9d6bf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:37.389329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 310/613] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:37.400867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901b00d-e2af-4a3e-9bb5-5089f1e9d6bf None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:37.460957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:37.478902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb7a03-eb88-45ff-88ca-8e7f69a159fd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:37.544475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b499245-5509-40b1-8fed-c2e4d17960ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:37.545705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499245-5509-40b1-8fed-c2e4d17960ef None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:37.546963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499245-5509-40b1-8fed-c2e4d17960ef None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:37.547504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b499245-5509-40b1-8fed-c2e4d17960ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:37.563393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b499245-5509-40b1-8fed-c2e4d17960ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-6b499245-5509-40b1-8fed-c2e4d17960ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:37.571847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:37.572930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:37.573786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:37.574767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901b00d-e2af-4a3e-9bb5-5089f1e9d6bf None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:37.576287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 304/614] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:37.583817 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b499245-5509-40b1-8fed-c2e4d17960ef None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:37.618799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3e885-eff8-4dfd-a9b2-9c743d586df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:37.619869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3e885-eff8-4dfd-a9b2-9c743d586df7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:37.620164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3e885-eff8-4dfd-a9b2-9c743d586df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:37.620447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3e885-eff8-4dfd-a9b2-9c743d586df7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:37.635316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3e885-eff8-4dfd-a9b2-9c743d586df7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:37.636791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3e885-eff8-4dfd-a9b2-9c743d586df7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:37.655846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1ba3e885-eff8-4dfd-a9b2-9c743d586df7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:37.666979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:37.667916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:37.669802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:37.669802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:37.669802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:37.669802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:37.669802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 311/615] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:37.675329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3e885-eff8-4dfd-a9b2-9c743d586df7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:37.787576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b499245-5509-40b1-8fed-c2e4d17960ef None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:37.875663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b959e5-ce5b-421c-9571-b5c343cfec57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:37.877206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b959e5-ce5b-421c-9571-b5c343cfec57 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:37.877719 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b959e5-ce5b-421c-9571-b5c343cfec57 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:37.878191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b959e5-ce5b-421c-9571-b5c343cfec57 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:37.879157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b959e5-ce5b-421c-9571-b5c343cfec57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:37.881142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b959e5-ce5b-421c-9571-b5c343cfec57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:37.882765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b959e5-ce5b-421c-9571-b5c343cfec57 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:37.912084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3e885-eff8-4dfd-a9b2-9c743d586df7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:37.913041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 305/616] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:37.920266 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b959e5-ce5b-421c-9571-b5c343cfec57 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:37.960245 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:38.046338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972e1e1-01fa-4a05-ae87-b31f64878bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:38.046952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972e1e1-01fa-4a05-ae87-b31f64878bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:38.047240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972e1e1-01fa-4a05-ae87-b31f64878bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:38.047519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972e1e1-01fa-4a05-ae87-b31f64878bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:38.048264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972e1e1-01fa-4a05-ae87-b31f64878bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:38.048573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972e1e1-01fa-4a05-ae87-b31f64878bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:38.051427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972e1e1-01fa-4a05-ae87-b31f64878bb8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:38.058596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b959e5-ce5b-421c-9571-b5c343cfec57 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:38.071508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 312/617] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:38.076789 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972e1e1-01fa-4a05-ae87-b31f64878bb8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:38.100340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7728e2f-bb89-45ad-b838-7378e709b6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:38.102995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7728e2f-bb89-45ad-b838-7378e709b6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:38.103419 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7728e2f-bb89-45ad-b838-7378e709b6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:38.103841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7728e2f-bb89-45ad-b838-7378e709b6e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:38.120167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7728e2f-bb89-45ad-b838-7378e709b6e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:38.124691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7728e2f-bb89-45ad-b838-7378e709b6e2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d7728e2f-bb89-45ad-b838-7378e709b6e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:38.154350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:38.155488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:38.156729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:38.156729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:38.156729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:38.156729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:38.165259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 306/618] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:38.171234 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7728e2f-bb89-45ad-b838-7378e709b6e2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:38.250023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:38.296209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:38.334884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7728e2f-bb89-45ad-b838-7378e709b6e2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:38.364417 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972e1e1-01fa-4a05-ae87-b31f64878bb8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:38.389276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44aa1f-666e-4cda-9020-c6d69aad153c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:38.391155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44aa1f-666e-4cda-9020-c6d69aad153c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:38.392917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44aa1f-666e-4cda-9020-c6d69aad153c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:38.393345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44aa1f-666e-4cda-9020-c6d69aad153c None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:38.394286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44aa1f-666e-4cda-9020-c6d69aad153c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:38.394876 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44aa1f-666e-4cda-9020-c6d69aad153c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:38.397475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44aa1f-666e-4cda-9020-c6d69aad153c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:38.432197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 307/619] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:38.439111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44aa1f-666e-4cda-9020-c6d69aad153c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:38.477000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c4e39-93f6-44b7-b393-27b883b0c373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:38.478265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c4e39-93f6-44b7-b393-27b883b0c373 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:38.478794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c4e39-93f6-44b7-b393-27b883b0c373 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:38.479275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c4e39-93f6-44b7-b393-27b883b0c373 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:38.492265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c4e39-93f6-44b7-b393-27b883b0c373 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:38.494304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c4e39-93f6-44b7-b393-27b883b0c373 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a62c4e39-93f6-44b7-b393-27b883b0c373 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:38.523601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:38.524606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:38.525589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:38.525589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:38.525589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:38.525589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:38.525589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:38.525589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:38.525589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:38.525589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:38.525589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:30:38.525589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:38.539180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 313/620] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:38.546178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c4e39-93f6-44b7-b393-27b883b0c373 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:38.636508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44aa1f-666e-4cda-9020-c6d69aad153c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:38.698345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:38.711395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c4e39-93f6-44b7-b393-27b883b0c373 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:38.714550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bf635-0efd-4b09-98e6-0c45f86ece4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:38.714550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bf635-0efd-4b09-98e6-0c45f86ece4a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:38.714731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bf635-0efd-4b09-98e6-0c45f86ece4a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:38.715027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bf635-0efd-4b09-98e6-0c45f86ece4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:38.730389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bf635-0efd-4b09-98e6-0c45f86ece4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:38.738243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bf635-0efd-4b09-98e6-0c45f86ece4a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:38.768370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-348bf635-0efd-4b09-98e6-0c45f86ece4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:38.773078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:38.784268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 308/621] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Jul 18 16:30:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849d1c4-a748-4e81-9cea-da3273121048 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:38.785313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849d1c4-a748-4e81-9cea-da3273121048 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:38.789273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849d1c4-a748-4e81-9cea-da3273121048 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:38.789273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 309/622] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:38.793761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849d1c4-a748-4e81-9cea-da3273121048 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:38.799858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f446bf-66ba-45ef-b346-2dd4a6b04d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:38.800547 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f446bf-66ba-45ef-b346-2dd4a6b04d90 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:38.800938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f446bf-66ba-45ef-b346-2dd4a6b04d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:38.801231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f446bf-66ba-45ef-b346-2dd4a6b04d90 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:38.802085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f446bf-66ba-45ef-b346-2dd4a6b04d90 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:38.802395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f446bf-66ba-45ef-b346-2dd4a6b04d90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:38.803850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f446bf-66ba-45ef-b346-2dd4a6b04d90 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:38.835175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 314/623] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Jul 18 16:30:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:38.836633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae453cd-c8db-4666-be55-841d88a58669 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:38.837055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae453cd-c8db-4666-be55-841d88a58669 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:38.837342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae453cd-c8db-4666-be55-841d88a58669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:38.863279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5ae453cd-c8db-4666-be55-841d88a58669 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:38.864493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5ae453cd-c8db-4666-be55-841d88a58669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:38.955162 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 315/624] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:38.955855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae453cd-c8db-4666-be55-841d88a58669 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:38.976085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849d1c4-a748-4e81-9cea-da3273121048 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:39.016679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22a88d-401e-4b34-a052-ccec7fcd43a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:39.017569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22a88d-401e-4b34-a052-ccec7fcd43a0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.017852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22a88d-401e-4b34-a052-ccec7fcd43a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.018122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22a88d-401e-4b34-a052-ccec7fcd43a0 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.020718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22a88d-401e-4b34-a052-ccec7fcd43a0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:39.021067 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22a88d-401e-4b34-a052-ccec7fcd43a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:39.022520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22a88d-401e-4b34-a052-ccec7fcd43a0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:39.047014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 310/625] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Jul 18 16:30:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:39.047014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02232b8b-f5f2-4a89-acfa-7a091042c536 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.047014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02232b8b-f5f2-4a89-acfa-7a091042c536 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.047555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02232b8b-f5f2-4a89-acfa-7a091042c536 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.049352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 311/626] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:39.050407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02232b8b-f5f2-4a89-acfa-7a091042c536 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:39.102159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae453cd-c8db-4666-be55-841d88a58669 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:39.114036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:39.123619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:39.145616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c51163-c04c-4dba-9d44-565d02e17229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:39.146427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c51163-c04c-4dba-9d44-565d02e17229 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.146712 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c51163-c04c-4dba-9d44-565d02e17229 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.146995 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c51163-c04c-4dba-9d44-565d02e17229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.163186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c51163-c04c-4dba-9d44-565d02e17229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:39.165334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c51163-c04c-4dba-9d44-565d02e17229 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:39.188206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02232b8b-f5f2-4a89-acfa-7a091042c536 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-89c51163-c04c-4dba-9d44-565d02e17229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:39.188377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:39.189421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:39.190304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:39.190304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:39.190304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:39.190304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:39.190304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 316/627] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:39.191429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c51163-c04c-4dba-9d44-565d02e17229 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:39.241064 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fed1d-5cd9-4132-a564-c2d1029a5fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:39.242119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fed1d-5cd9-4132-a564-c2d1029a5fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.242846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fed1d-5cd9-4132-a564-c2d1029a5fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.243437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fed1d-5cd9-4132-a564-c2d1029a5fe0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.259766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fed1d-5cd9-4132-a564-c2d1029a5fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-505fed1d-5cd9-4132-a564-c2d1029a5fe0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:39.270752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:39.272023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:39.273088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:39.277237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 312/628] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:39.279462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c44049-3a86-4dd9-8e68-489030809e28 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.279754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c44049-3a86-4dd9-8e68-489030809e28 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.280033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c44049-3a86-4dd9-8e68-489030809e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.299913 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e7c44049-3a86-4dd9-8e68-489030809e28 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:39.302287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e7c44049-3a86-4dd9-8e68-489030809e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:39.328207 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c51163-c04c-4dba-9d44-565d02e17229 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:39.377975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c7b55-f6d9-4a07-8587-b816e3c93ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:39.378686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c7b55-f6d9-4a07-8587-b816e3c93ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.378976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c7b55-f6d9-4a07-8587-b816e3c93ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.381210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c7b55-f6d9-4a07-8587-b816e3c93ef7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.402491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 313/629] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:39.404228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c44049-3a86-4dd9-8e68-489030809e28 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:39.441124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:39.459568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:39.493113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c7b55-f6d9-4a07-8587-b816e3c93ef7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:39.495911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c7b55-f6d9-4a07-8587-b816e3c93ef7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:39.539791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c44049-3a86-4dd9-8e68-489030809e28 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:39.590634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533b8c1-56fe-4203-b5cb-405d9950a0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:39.592109 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533b8c1-56fe-4203-b5cb-405d9950a0ff None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.592481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533b8c1-56fe-4203-b5cb-405d9950a0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.592829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533b8c1-56fe-4203-b5cb-405d9950a0ff None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.593820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533b8c1-56fe-4203-b5cb-405d9950a0ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:39.594197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533b8c1-56fe-4203-b5cb-405d9950a0ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:39.595876 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533b8c1-56fe-4203-b5cb-405d9950a0ff None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:39.612072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 314/630] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:39.613321 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533b8c1-56fe-4203-b5cb-405d9950a0ff None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:39.646988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 317/631] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:39.647865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c7b55-f6d9-4a07-8587-b816e3c93ef7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:39.723205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:39.751589 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:39.755796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533b8c1-56fe-4203-b5cb-405d9950a0ff None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:39.773310 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c7b55-f6d9-4a07-8587-b816e3c93ef7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:39.797418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4565ac-c299-40e5-9070-ff734ec71a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:39.798274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4565ac-c299-40e5-9070-ff734ec71a00 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.798564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4565ac-c299-40e5-9070-ff734ec71a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.798848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4565ac-c299-40e5-9070-ff734ec71a00 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.799828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4565ac-c299-40e5-9070-ff734ec71a00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:39.800143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4565ac-c299-40e5-9070-ff734ec71a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:39.801581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4565ac-c299-40e5-9070-ff734ec71a00 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:39.811632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 315/632] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:39.812367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4565ac-c299-40e5-9070-ff734ec71a00 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:39.820521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaf460-9450-42b3-97ba-c6d46814be79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:39.821260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaf460-9450-42b3-97ba-c6d46814be79 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.821544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaf460-9450-42b3-97ba-c6d46814be79 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.821829 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaf460-9450-42b3-97ba-c6d46814be79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.834892 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadaf460-9450-42b3-97ba-c6d46814be79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-dadaf460-9450-42b3-97ba-c6d46814be79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:39.843595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:39.844572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:39.845466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 318/633] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:39.846152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaf460-9450-42b3-97ba-c6d46814be79 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:39.941418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4565ac-c299-40e5-9070-ff734ec71a00 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:39.960795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaf460-9450-42b3-97ba-c6d46814be79 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:39.979847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f49180-200b-4f63-ae34-8aa718e4948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:39.981351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f49180-200b-4f63-ae34-8aa718e4948d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.981890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:39.982945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f49180-200b-4f63-ae34-8aa718e4948d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.983479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f49180-200b-4f63-ae34-8aa718e4948d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:39.990966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f49180-200b-4f63-ae34-8aa718e4948d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:39.992702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f49180-200b-4f63-ae34-8aa718e4948d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:39.996755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3051f4-227c-4495-a5c1-f793fea78130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:39.997643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3051f4-227c-4495-a5c1-f793fea78130 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:39.997926 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3051f4-227c-4495-a5c1-f793fea78130 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:39.998206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3051f4-227c-4495-a5c1-f793fea78130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.009594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3051f4-227c-4495-a5c1-f793fea78130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:40.011095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3051f4-227c-4495-a5c1-f793fea78130 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c2f49180-200b-4f63-ae34-8aa718e4948d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:40.012408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:40.014078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:40.015200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:40.015200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:40.015200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:40.015200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:40.015200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:40.015200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:40.015200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:40.015200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:40.015200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 316/634] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:40.016617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931d4d-28d3-47d5-88ac-ad6ffda2c806 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.017219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931d4d-28d3-47d5-88ac-ad6ffda2c806 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.017468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931d4d-28d3-47d5-88ac-ad6ffda2c806 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.022633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 317/635] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:40.022633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67931d4d-28d3-47d5-88ac-ad6ffda2c806 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-1e3051f4-227c-4495-a5c1-f793fea78130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:40.035143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:40.036937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:40.039250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:40.039250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:40.039250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:40.039250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:40.041108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 319/636] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:40.042424 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3051f4-227c-4495-a5c1-f793fea78130 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:40.045964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:40.138487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67931d4d-28d3-47d5-88ac-ad6ffda2c806 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:40.176957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3dbaa-78a4-4858-b946-620e8d1acecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:40.177857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3051f4-227c-4495-a5c1-f793fea78130 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:40.177968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3dbaa-78a4-4858-b946-620e8d1acecf None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.178223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3dbaa-78a4-4858-b946-620e8d1acecf None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.178665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3dbaa-78a4-4858-b946-620e8d1acecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.190558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3dbaa-78a4-4858-b946-620e8d1acecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:40.192158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3dbaa-78a4-4858-b946-620e8d1acecf None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3ea3dbaa-78a4-4858-b946-620e8d1acecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:40.213455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:40.214387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:40.215284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:40.215284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:40.215284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:40.215284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:40.215284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 318/637] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:40.215564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3dbaa-78a4-4858-b946-620e8d1acecf None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:40.220269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c2f2e-e450-47e0-bad8-a02624e2ee99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:40.221118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c2f2e-e450-47e0-bad8-a02624e2ee99 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.221406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c2f2e-e450-47e0-bad8-a02624e2ee99 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.221687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c2f2e-e450-47e0-bad8-a02624e2ee99 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.222462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c2f2e-e450-47e0-bad8-a02624e2ee99 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:40.222768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c2f2e-e450-47e0-bad8-a02624e2ee99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:40.224201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c2f2e-e450-47e0-bad8-a02624e2ee99 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:40.233740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 320/638] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:40.234857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a2a4a-119a-4a97-9364-26cbf1a4eaa0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.235105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a2a4a-119a-4a97-9364-26cbf1a4eaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.235578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a2a4a-119a-4a97-9364-26cbf1a4eaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.255133 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a23a2a4a-119a-4a97-9364-26cbf1a4eaa0 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:40.256379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a23a2a4a-119a-4a97-9364-26cbf1a4eaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:40.289668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:40.305572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:40.332293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 321/639] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:40.333533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a2a4a-119a-4a97-9364-26cbf1a4eaa0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:40.336467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3dbaa-78a4-4858-b946-620e8d1acecf None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:40.379303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814c6a3-0d08-4820-837d-8ead3e0e62aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:40.379968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c6a3-0d08-4820-837d-8ead3e0e62aa None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.380254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c6a3-0d08-4820-837d-8ead3e0e62aa None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.380528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814c6a3-0d08-4820-837d-8ead3e0e62aa None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.381379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814c6a3-0d08-4820-837d-8ead3e0e62aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:40.381691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814c6a3-0d08-4820-837d-8ead3e0e62aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:40.383143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814c6a3-0d08-4820-837d-8ead3e0e62aa None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:40.397229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 319/640] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:40.398071 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814c6a3-0d08-4820-837d-8ead3e0e62aa None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:40.489094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a2a4a-119a-4a97-9364-26cbf1a4eaa0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:40.529775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83372d-a2c3-4bc6-9753-52afb16c97ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:40.530800 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83372d-a2c3-4bc6-9753-52afb16c97ab None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.531247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83372d-a2c3-4bc6-9753-52afb16c97ab None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.531642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83372d-a2c3-4bc6-9753-52afb16c97ab None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.532266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83372d-a2c3-4bc6-9753-52afb16c97ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:40.532597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83372d-a2c3-4bc6-9753-52afb16c97ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:40.533994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83372d-a2c3-4bc6-9753-52afb16c97ab None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:40.536270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814c6a3-0d08-4820-837d-8ead3e0e62aa None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:40.546508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 322/641] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:40.552039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83372d-a2c3-4bc6-9753-52afb16c97ab None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:40.573162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:40.581216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1679e-a527-496d-a7a6-fd7bc4e43c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:40.581595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:40.581894 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1679e-a527-496d-a7a6-fd7bc4e43c48 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.582276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1679e-a527-496d-a7a6-fd7bc4e43c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.582456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1679e-a527-496d-a7a6-fd7bc4e43c48 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.583193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1679e-a527-496d-a7a6-fd7bc4e43c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:40.583502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1679e-a527-496d-a7a6-fd7bc4e43c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:40.584920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1679e-a527-496d-a7a6-fd7bc4e43c48 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:40.597779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 320/642] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:40.598578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1679e-a527-496d-a7a6-fd7bc4e43c48 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:40.669804 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83372d-a2c3-4bc6-9753-52afb16c97ab None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:40.706645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e8ca8-4f50-46ab-a6e5-3d4b2f5abcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:40.707451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e8ca8-4f50-46ab-a6e5-3d4b2f5abcef None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.707742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e8ca8-4f50-46ab-a6e5-3d4b2f5abcef None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.708016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e8ca8-4f50-46ab-a6e5-3d4b2f5abcef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.710739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1679e-a527-496d-a7a6-fd7bc4e43c48 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:40.721846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e8ca8-4f50-46ab-a6e5-3d4b2f5abcef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:40.723584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e8ca8-4f50-46ab-a6e5-3d4b2f5abcef None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-972e8ca8-4f50-46ab-a6e5-3d4b2f5abcef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:40.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:40.745742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:40.746850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:40.746850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:40.746850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:40.746850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:40.748525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 323/643] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:40.749525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e8ca8-4f50-46ab-a6e5-3d4b2f5abcef None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:40.753806 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60583b-d2aa-4feb-a81e-acb7f2dd6cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:40.754745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60583b-d2aa-4feb-a81e-acb7f2dd6cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.755104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60583b-d2aa-4feb-a81e-acb7f2dd6cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.755446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60583b-d2aa-4feb-a81e-acb7f2dd6cc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.774604 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60583b-d2aa-4feb-a81e-acb7f2dd6cc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ef60583b-d2aa-4feb-a81e-acb7f2dd6cc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:40.783114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:40.784687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 321/644] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:40.785761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60583b-d2aa-4feb-a81e-acb7f2dd6cc3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:40.860362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:40.870906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:40.896608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e8ca8-4f50-46ab-a6e5-3d4b2f5abcef None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:40.912627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60583b-d2aa-4feb-a81e-acb7f2dd6cc3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:40.941253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e55a0-f265-45d2-b14a-94cd6d57816b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:40.942403 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e55a0-f265-45d2-b14a-94cd6d57816b None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.942688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e55a0-f265-45d2-b14a-94cd6d57816b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.942966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e55a0-f265-45d2-b14a-94cd6d57816b None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/bf341f1bb8944a3e9331678100b941a1/roles/3fc59073330c4557ad178629386b0852` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.953923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540ba5-8a41-4025-9f8c-7b3ec0e5abac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:40.954988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540ba5-8a41-4025-9f8c-7b3ec0e5abac None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:40.955450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540ba5-8a41-4025-9f8c-7b3ec0e5abac None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:40.955724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94540ba5-8a41-4025-9f8c-7b3ec0e5abac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:40.959255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e55a0-f265-45d2-b14a-94cd6d57816b None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=bf341f1bb8944a3e9331678100b941a1, role_id=3fc59073330c4557ad178629386b0852)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:40.960721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e55a0-f265-45d2-b14a-94cd6d57816b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:40.963280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540ba5-8a41-4025-9f8c-7b3ec0e5abac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:40.965089 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94540ba5-8a41-4025-9f8c-7b3ec0e5abac None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-94540ba5-8a41-4025-9f8c-7b3ec0e5abac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:40.980025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:40.981556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:40.982445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:40.982445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:40.982445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:40.982445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:40.982445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:40.982445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:40.982445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:40.982445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:40.982445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 322/645] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:40.982445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540ba5-8a41-4025-9f8c-7b3ec0e5abac None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:41.025128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 324/646] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:30:40 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/bf341f1bb8944a3e9331678100b941a1/roles/3fc59073330c4557ad178629386b0852 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:30:41.026044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e55a0-f265-45d2-b14a-94cd6d57816b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:41.106102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94540ba5-8a41-4025-9f8c-7b3ec0e5abac None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:41.132693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:41.151675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f88a9-c77a-4862-a38b-cff26a7e2636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:41.154130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f88a9-c77a-4862-a38b-cff26a7e2636 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:41.154130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f88a9-c77a-4862-a38b-cff26a7e2636 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:41.154130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f88a9-c77a-4862-a38b-cff26a7e2636 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:41.162051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f88a9-c77a-4862-a38b-cff26a7e2636 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:41.163615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f88a9-c77a-4862-a38b-cff26a7e2636 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-140f88a9-c77a-4862-a38b-cff26a7e2636 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:41.189223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:41.190223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:41.191181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:41.191181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:41.191181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:41.191181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:41.191181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:41.191181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:41.191181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:41.191181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:41.191181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 16:30:41.191181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:41.191718 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e55a0-f265-45d2-b14a-94cd6d57816b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:41.196191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 323/647] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:41.198028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f88a9-c77a-4862-a38b-cff26a7e2636 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:41.238705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:41.251426 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f7a78-a1dc-4d66-8016-4135e022518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:41.252102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f7a78-a1dc-4d66-8016-4135e022518c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:41.252389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f7a78-a1dc-4d66-8016-4135e022518c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:41.254028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f7a78-a1dc-4d66-8016-4135e022518c None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:41.254028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f7a78-a1dc-4d66-8016-4135e022518c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:41.254028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f7a78-a1dc-4d66-8016-4135e022518c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:41.255393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f7a78-a1dc-4d66-8016-4135e022518c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:41.270521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 325/648] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:41.271608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f7a78-a1dc-4d66-8016-4135e022518c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:41.333911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f88a9-c77a-4862-a38b-cff26a7e2636 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:41.386987 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408a05a-d612-4e4b-86e4-0c5e3f8a02f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:41.388224 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a05a-d612-4e4b-86e4-0c5e3f8a02f8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:41.388515 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a05a-d612-4e4b-86e4-0c5e3f8a02f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:41.388927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408a05a-d612-4e4b-86e4-0c5e3f8a02f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:41.389684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408a05a-d612-4e4b-86e4-0c5e3f8a02f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:41.389993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408a05a-d612-4e4b-86e4-0c5e3f8a02f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:41.391380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408a05a-d612-4e4b-86e4-0c5e3f8a02f8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:41.405805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 324/649] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:41.406779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408a05a-d612-4e4b-86e4-0c5e3f8a02f8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:41.414310 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f7a78-a1dc-4d66-8016-4135e022518c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:41.452254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:41.464391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c38a9-a145-4522-a24e-56c86a9cfad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:41.465159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c38a9-a145-4522-a24e-56c86a9cfad1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:41.465442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c38a9-a145-4522-a24e-56c86a9cfad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:41.466405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c38a9-a145-4522-a24e-56c86a9cfad1 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:41.467206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c38a9-a145-4522-a24e-56c86a9cfad1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:41.467515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c38a9-a145-4522-a24e-56c86a9cfad1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:41.469061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c38a9-a145-4522-a24e-56c86a9cfad1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:41.489749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 326/650] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:41.490729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c38a9-a145-4522-a24e-56c86a9cfad1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:41.543625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408a05a-d612-4e4b-86e4-0c5e3f8a02f8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:41.560727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:41.593474 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce724f-905e-4de2-8e4d-570bf84f49fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:41.594956 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce724f-905e-4de2-8e4d-570bf84f49fb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:41.595274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce724f-905e-4de2-8e4d-570bf84f49fb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:41.595552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce724f-905e-4de2-8e4d-570bf84f49fb None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:41.596403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce724f-905e-4de2-8e4d-570bf84f49fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:41.596713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce724f-905e-4de2-8e4d-570bf84f49fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:41.598287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce724f-905e-4de2-8e4d-570bf84f49fb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:41.628591 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 325/651] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Jul 18 16:30:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:41.628591 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce724f-905e-4de2-8e4d-570bf84f49fb None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:41.642964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c38a9-a145-4522-a24e-56c86a9cfad1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:41.691395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae6f99-f15e-412c-98e5-dcc53bf76205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:41.692660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae6f99-f15e-412c-98e5-dcc53bf76205 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:41.693428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae6f99-f15e-412c-98e5-dcc53bf76205 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:41.693858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae6f99-f15e-412c-98e5-dcc53bf76205 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:41.704546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae6f99-f15e-412c-98e5-dcc53bf76205 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:41.706573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae6f99-f15e-412c-98e5-dcc53bf76205 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-dfae6f99-f15e-412c-98e5-dcc53bf76205 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:41.724116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:41.725652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:41.726660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:41.726660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:41.726660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:41.726660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:41.726660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:41.726660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:41.726660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:41.726660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:41.726660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 16:30:41.726660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:41.734590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 327/652] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:41.735959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae6f99-f15e-412c-98e5-dcc53bf76205 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:41.748139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:41.759687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce724f-905e-4de2-8e4d-570bf84f49fb None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:41.805991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b40ebb-f6d1-41cf-8efe-02b0fb0014f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:41.806800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b40ebb-f6d1-41cf-8efe-02b0fb0014f9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:41.807375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b40ebb-f6d1-41cf-8efe-02b0fb0014f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:41.807652 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b40ebb-f6d1-41cf-8efe-02b0fb0014f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:41.818762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b40ebb-f6d1-41cf-8efe-02b0fb0014f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:41.820706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b40ebb-f6d1-41cf-8efe-02b0fb0014f9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d9b40ebb-f6d1-41cf-8efe-02b0fb0014f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:41.843785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:41.845191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:41.846373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:41.846373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:41.846373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:41.846373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:41.852197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 326/653] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:41.852197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b40ebb-f6d1-41cf-8efe-02b0fb0014f9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:41.875361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae6f99-f15e-412c-98e5-dcc53bf76205 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:41.885803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:41.928325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4a711-1734-4aee-a1b2-151fd16a6d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:41.928325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4a711-1734-4aee-a1b2-151fd16a6d17 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:41.928325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4a711-1734-4aee-a1b2-151fd16a6d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:41.928325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4a711-1734-4aee-a1b2-151fd16a6d17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:41.928734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4a711-1734-4aee-a1b2-151fd16a6d17 None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=69ee145599384c6da2cba85a42832dcd) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:41.928734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4a711-1734-4aee-a1b2-151fd16a6d17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:41.930364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4a711-1734-4aee-a1b2-151fd16a6d17 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:41.941337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 328/654] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:41 2022] GET /identity/v3/role_assignments?user.id=69ee145599384c6da2cba85a42832dcd&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:41.944245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4a711-1734-4aee-a1b2-151fd16a6d17 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:42.008149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b40ebb-f6d1-41cf-8efe-02b0fb0014f9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:42.085288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d75940-c3bd-4849-ae32-892f5b0462e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:42.086135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d75940-c3bd-4849-ae32-892f5b0462e8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:42.090175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d75940-c3bd-4849-ae32-892f5b0462e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:42.090771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d75940-c3bd-4849-ae32-892f5b0462e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:42.101546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4a711-1734-4aee-a1b2-151fd16a6d17 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:42.123667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:42.151978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cc4ef-7f32-49b0-869a-4c5c1479cda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:42.152359 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cc4ef-7f32-49b0-869a-4c5c1479cda7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:42.152531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cc4ef-7f32-49b0-869a-4c5c1479cda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:42.152808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cc4ef-7f32-49b0-869a-4c5c1479cda7 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:42.154323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cc4ef-7f32-49b0-869a-4c5c1479cda7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:42.154640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cc4ef-7f32-49b0-869a-4c5c1479cda7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:42.156073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cc4ef-7f32-49b0-869a-4c5c1479cda7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:42.188652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 329/655] 10.222.0.88 () {56 vars in 1139 bytes} [Mon Jul 18 16:30:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:42.189588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cc4ef-7f32-49b0-869a-4c5c1479cda7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:42.230522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:42.234092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d75940-c3bd-4849-ae32-892f5b0462e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:42.235373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d75940-c3bd-4849-ae32-892f5b0462e8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:42.339543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cc4ef-7f32-49b0-869a-4c5c1479cda7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:42.377050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 327/656] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:42.377659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d75940-c3bd-4849-ae32-892f5b0462e8 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:42.404414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73945cdb-76f3-4922-b9ba-c936e3f7a248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:42.405755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73945cdb-76f3-4922-b9ba-c936e3f7a248 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:42.406707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73945cdb-76f3-4922-b9ba-c936e3f7a248 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:42.407192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73945cdb-76f3-4922-b9ba-c936e3f7a248 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:42.413813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73945cdb-76f3-4922-b9ba-c936e3f7a248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:42.414399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73945cdb-76f3-4922-b9ba-c936e3f7a248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:42.416048 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73945cdb-76f3-4922-b9ba-c936e3f7a248 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:42.435781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 330/657] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:42.436721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73945cdb-76f3-4922-b9ba-c936e3f7a248 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:42.451483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:42.561657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d75940-c3bd-4849-ae32-892f5b0462e8 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:42.570945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:42.585861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73945cdb-76f3-4922-b9ba-c936e3f7a248 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:42.645394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecee56-7e0a-4036-88ed-8015b5b0afae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:42.645994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecee56-7e0a-4036-88ed-8015b5b0afae None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:42.646279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecee56-7e0a-4036-88ed-8015b5b0afae None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:42.647068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecee56-7e0a-4036-88ed-8015b5b0afae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:42.651108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c4341-5c72-4a4c-a0d6-fdd4f59e4d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:42.651916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c4341-5c72-4a4c-a0d6-fdd4f59e4d4a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:42.652257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c4341-5c72-4a4c-a0d6-fdd4f59e4d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:42.652541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c4341-5c72-4a4c-a0d6-fdd4f59e4d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:42.679178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c4341-5c72-4a4c-a0d6-fdd4f59e4d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:42.680653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c4341-5c72-4a4c-a0d6-fdd4f59e4d4a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-b79c4341-5c72-4a4c-a0d6-fdd4f59e4d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:42.716914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:42.717856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:42.718738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:42.718738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:42.718738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:42.718738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:42.730877 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 331/658] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:42.730877 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c4341-5c72-4a4c-a0d6-fdd4f59e4d4a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:42.759916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecee56-7e0a-4036-88ed-8015b5b0afae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:42.764105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ecee56-7e0a-4036-88ed-8015b5b0afae None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:42.835722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:42.919512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c4341-5c72-4a4c-a0d6-fdd4f59e4d4a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:42.933082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 328/659] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:42.933911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecee56-7e0a-4036-88ed-8015b5b0afae None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:42.986592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7b6d0-195b-4d76-bed5-b30a9942aa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:42.987785 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7b6d0-195b-4d76-bed5-b30a9942aa5e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:42.988273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7b6d0-195b-4d76-bed5-b30a9942aa5e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:42.988841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7b6d0-195b-4d76-bed5-b30a9942aa5e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:43.000375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7b6d0-195b-4d76-bed5-b30a9942aa5e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:43.002221 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7b6d0-195b-4d76-bed5-b30a9942aa5e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:43.005247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-59b7b6d0-195b-4d76-bed5-b30a9942aa5e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:43.030749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:43.031855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:43.032792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:43.032792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:43.032792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:43.032792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:43.032792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:43.032792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:43.032792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:43.032792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:43.032792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:30:43.032792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:43.034976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 332/660] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:43.035956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7b6d0-195b-4d76-bed5-b30a9942aa5e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:43.069874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecee56-7e0a-4036-88ed-8015b5b0afae None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:43.110290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bfee4-a340-4b6e-9af9-601d401aa822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:43.112135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bfee4-a340-4b6e-9af9-601d401aa822 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:43.112747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bfee4-a340-4b6e-9af9-601d401aa822 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:43.114546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bfee4-a340-4b6e-9af9-601d401aa822 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:43.116113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46bfee4-a340-4b6e-9af9-601d401aa822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:43.116766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46bfee4-a340-4b6e-9af9-601d401aa822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:43.119355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46bfee4-a340-4b6e-9af9-601d401aa822 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:43.141754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 329/661] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:43.143561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bfee4-a340-4b6e-9af9-601d401aa822 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:43.150835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:43.196656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7b6d0-195b-4d76-bed5-b30a9942aa5e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:43.242304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6793f-c4a5-4b41-84c0-4b369d7e71e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:43.243990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6793f-c4a5-4b41-84c0-4b369d7e71e0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:43.244473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6793f-c4a5-4b41-84c0-4b369d7e71e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:43.244969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6793f-c4a5-4b41-84c0-4b369d7e71e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:43.259429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6793f-c4a5-4b41-84c0-4b369d7e71e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:43.263383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6793f-c4a5-4b41-84c0-4b369d7e71e0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:43.274550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bfee4-a340-4b6e-9af9-601d401aa822 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f6a6793f-c4a5-4b41-84c0-4b369d7e71e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:43.284700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:43.285738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:43.286706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:43.286706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:43.286706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:43.286706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:43.289111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 333/662] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:43.290900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa31f43-ce67-435a-8ab9-0e60d2d8e17c None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:43.290900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa31f43-ce67-435a-8ab9-0e60d2d8e17c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:43.291200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa31f43-ce67-435a-8ab9-0e60d2d8e17c None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:43.297008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 334/663] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:43.297008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa31f43-ce67-435a-8ab9-0e60d2d8e17c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:43.329390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4792493-3cf4-44fd-9897-5bcde5895395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:43.330587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4792493-3cf4-44fd-9897-5bcde5895395 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:43.330969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4792493-3cf4-44fd-9897-5bcde5895395 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:43.331339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4792493-3cf4-44fd-9897-5bcde5895395 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:43.332275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4792493-3cf4-44fd-9897-5bcde5895395 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:43.332680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4792493-3cf4-44fd-9897-5bcde5895395 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:43.334455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4792493-3cf4-44fd-9897-5bcde5895395 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:43.361250 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 330/664] 10.222.0.88 () {56 vars in 1137 bytes} [Mon Jul 18 16:30:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:43.362463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4792493-3cf4-44fd-9897-5bcde5895395 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:43.368162 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:43.435449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa31f43-ce67-435a-8ab9-0e60d2d8e17c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:43.437769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:43.482240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f157e-2de9-4abe-a9bd-ccb48a998ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:43.482999 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f157e-2de9-4abe-a9bd-ccb48a998ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:43.483289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f157e-2de9-4abe-a9bd-ccb48a998ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:43.483569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f157e-2de9-4abe-a9bd-ccb48a998ee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:43.489281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4792493-3cf4-44fd-9897-5bcde5895395 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:43.491866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f157e-2de9-4abe-a9bd-ccb48a998ee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:43.493650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f157e-2de9-4abe-a9bd-ccb48a998ee5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-862f157e-2de9-4abe-a9bd-ccb48a998ee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:43.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:43.510683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:43.511601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:43.511601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:43.511601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:43.511601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:43.511601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:43.511601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:43.511601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:43.511601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:43.516271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 335/665] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:43.517650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b36d9-d8a1-4fe4-b015-37f1357cf7ed None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:43.517939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b36d9-d8a1-4fe4-b015-37f1357cf7ed None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:43.518220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b36d9-d8a1-4fe4-b015-37f1357cf7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:43.533395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04bd0b-782c-4d7d-8d50-1c29adf3dbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:43.534081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04bd0b-782c-4d7d-8d50-1c29adf3dbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:43.534366 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04bd0b-782c-4d7d-8d50-1c29adf3dbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:43.535137 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04bd0b-782c-4d7d-8d50-1c29adf3dbb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:43.535882 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04bd0b-782c-4d7d-8d50-1c29adf3dbb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:43.536186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04bd0b-782c-4d7d-8d50-1c29adf3dbb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:43.538040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04bd0b-782c-4d7d-8d50-1c29adf3dbb2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:43.542654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-685b36d9-d8a1-4fe4-b015-37f1357cf7ed None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:43.544108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-685b36d9-d8a1-4fe4-b015-37f1357cf7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:43.557032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 331/666] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:43.558296 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04bd0b-782c-4d7d-8d50-1c29adf3dbb2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:43.647322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 336/667] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:43.647322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b36d9-d8a1-4fe4-b015-37f1357cf7ed None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:43.695583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:43.720420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04bd0b-782c-4d7d-8d50-1c29adf3dbb2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:43.747734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:43.768874 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b36d9-d8a1-4fe4-b015-37f1357cf7ed None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:43.770200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf462e-78c3-4dc8-9ca3-bbdd1f91e642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:43.771209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf462e-78c3-4dc8-9ca3-bbdd1f91e642 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:43.772474 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf462e-78c3-4dc8-9ca3-bbdd1f91e642 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:43.772766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf462e-78c3-4dc8-9ca3-bbdd1f91e642 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:43.814898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280053e-c427-4ebe-8045-711573a3200e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:43.815562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280053e-c427-4ebe-8045-711573a3200e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:43.816110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280053e-c427-4ebe-8045-711573a3200e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:43.816353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280053e-c427-4ebe-8045-711573a3200e None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:43.817224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280053e-c427-4ebe-8045-711573a3200e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:43.818018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280053e-c427-4ebe-8045-711573a3200e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:43.819435 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280053e-c427-4ebe-8045-711573a3200e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:43.839898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 337/668] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:43.841508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280053e-c427-4ebe-8045-711573a3200e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:43.869072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf462e-78c3-4dc8-9ca3-bbdd1f91e642 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:43.870863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf462e-78c3-4dc8-9ca3-bbdd1f91e642 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:43.989524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280053e-c427-4ebe-8045-711573a3200e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:43.991739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:44.022234 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 332/669] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:44.029047 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf462e-78c3-4dc8-9ca3-bbdd1f91e642 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:44.032565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d7f90-9705-433a-93c1-868e5e671f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:44.033449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d7f90-9705-433a-93c1-868e5e671f1f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:44.034160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d7f90-9705-433a-93c1-868e5e671f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:44.034444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d7f90-9705-433a-93c1-868e5e671f1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:44.047014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d7f90-9705-433a-93c1-868e5e671f1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:44.048656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d7f90-9705-433a-93c1-868e5e671f1f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:44.061304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bb5d7f90-9705-433a-93c1-868e5e671f1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:44.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:44.076836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:44.078639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:44.078639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:44.078639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:44.078639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:44.078639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:44.078639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:44.078639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:44.078639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:44.078639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 338/670] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:44.078639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d7f90-9705-433a-93c1-868e5e671f1f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:44.165662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf462e-78c3-4dc8-9ca3-bbdd1f91e642 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:44.212592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b609977-a561-4372-9607-238b915ee49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:44.213470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b609977-a561-4372-9607-238b915ee49b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:44.213976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b609977-a561-4372-9607-238b915ee49b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:44.214094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b609977-a561-4372-9607-238b915ee49b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:44.223212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d7f90-9705-433a-93c1-868e5e671f1f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:44.228008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b609977-a561-4372-9607-238b915ee49b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4b609977-a561-4372-9607-238b915ee49b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:44.235910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:44.236911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:44.237764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:44.242718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 333/671] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:44.243619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b609977-a561-4372-9607-238b915ee49b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:44.265554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b6bf7-66ab-4709-b087-fa2fae5931d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:44.266216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b6bf7-66ab-4709-b087-fa2fae5931d0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:44.266496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b6bf7-66ab-4709-b087-fa2fae5931d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:44.266774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b6bf7-66ab-4709-b087-fa2fae5931d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:44.267752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b6bf7-66ab-4709-b087-fa2fae5931d0 None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=54c7869983bf4534b3e730af0185eac7) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:44.268053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b6bf7-66ab-4709-b087-fa2fae5931d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:44.269521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b6bf7-66ab-4709-b087-fa2fae5931d0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:44.276434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 339/672] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:44 2022] GET /identity/v3/role_assignments?user.id=54c7869983bf4534b3e730af0185eac7&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:44.278080 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b6bf7-66ab-4709-b087-fa2fae5931d0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:44.316274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:44.351877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:44.379581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b609977-a561-4372-9607-238b915ee49b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:44.421416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b6bf7-66ab-4709-b087-fa2fae5931d0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:44.425848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798ca72-ccd7-4b2f-928e-4afbe58f48bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:44.426915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798ca72-ccd7-4b2f-928e-4afbe58f48bc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:44.427201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798ca72-ccd7-4b2f-928e-4afbe58f48bc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:44.427487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798ca72-ccd7-4b2f-928e-4afbe58f48bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:44.440284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798ca72-ccd7-4b2f-928e-4afbe58f48bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:44.442037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798ca72-ccd7-4b2f-928e-4afbe58f48bc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-0798ca72-ccd7-4b2f-928e-4afbe58f48bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:44.470057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:44.471007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:44.471929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:44.471929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:44.471929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:44.471929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:44.473152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f895e-ddf8-4d6e-b6f0-1d1826f0ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:44.473833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f895e-ddf8-4d6e-b6f0-1d1826f0ff70 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:44.479503 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 334/673] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:44.479690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f895e-ddf8-4d6e-b6f0-1d1826f0ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:44.479690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f895e-ddf8-4d6e-b6f0-1d1826f0ff70 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:44.479831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678ca90-3b45-41ac-bbcd-ef7d1dc01bfe None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:44.479831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678ca90-3b45-41ac-bbcd-ef7d1dc01bfe None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:44.479831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678ca90-3b45-41ac-bbcd-ef7d1dc01bfe None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:44.480027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f895e-ddf8-4d6e-b6f0-1d1826f0ff70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:44.480027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f895e-ddf8-4d6e-b6f0-1d1826f0ff70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:44.480027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f895e-ddf8-4d6e-b6f0-1d1826f0ff70 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:44.482151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 335/674] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:44.483389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678ca90-3b45-41ac-bbcd-ef7d1dc01bfe None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:44.495093 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 340/675] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:44.496230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f895e-ddf8-4d6e-b6f0-1d1826f0ff70 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:44.617134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:44.624279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678ca90-3b45-41ac-bbcd-ef7d1dc01bfe None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:44.634766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f895e-ddf8-4d6e-b6f0-1d1826f0ff70 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:44.675210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e1de5-d671-4662-9a07-df927c2eb0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:44.676050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e1de5-d671-4662-9a07-df927c2eb0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:44.676432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e1de5-d671-4662-9a07-df927c2eb0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:44.676707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e1de5-d671-4662-9a07-df927c2eb0b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:44.677649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e1de5-d671-4662-9a07-df927c2eb0b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:44.677955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e1de5-d671-4662-9a07-df927c2eb0b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:44.679702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e1de5-d671-4662-9a07-df927c2eb0b6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:44.680265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869191b2-5d10-4485-a233-486a0d5fd592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:44.681220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869191b2-5d10-4485-a233-486a0d5fd592 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:44.681352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:44.681528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869191b2-5d10-4485-a233-486a0d5fd592 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:44.681783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869191b2-5d10-4485-a233-486a0d5fd592 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:44.682526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869191b2-5d10-4485-a233-486a0d5fd592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:44.682838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869191b2-5d10-4485-a233-486a0d5fd592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:44.685457 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869191b2-5d10-4485-a233-486a0d5fd592 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:44.695070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 336/676] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:44.696501 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756a55e-302b-4cf1-a92d-335bf89f94fb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:44.696797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756a55e-302b-4cf1-a92d-335bf89f94fb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:44.697672 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756a55e-302b-4cf1-a92d-335bf89f94fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:44.704817 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 341/677] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:44.706750 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869191b2-5d10-4485-a233-486a0d5fd592 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:44.721953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1756a55e-302b-4cf1-a92d-335bf89f94fb None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:44.723342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1756a55e-302b-4cf1-a92d-335bf89f94fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:44.813701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 337/678] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:44.814631 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756a55e-302b-4cf1-a92d-335bf89f94fb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:44.846129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869191b2-5d10-4485-a233-486a0d5fd592 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:44.893936 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f32f6-45e5-474a-8d37-c0867ad9edb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:44.894818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f32f6-45e5-474a-8d37-c0867ad9edb8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:44.895230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f32f6-45e5-474a-8d37-c0867ad9edb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:44.895525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f32f6-45e5-474a-8d37-c0867ad9edb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:44.896379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f32f6-45e5-474a-8d37-c0867ad9edb8 None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=de8484c3b114467ea59bebb2460ed546) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:44.896988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f32f6-45e5-474a-8d37-c0867ad9edb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:44.898981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f32f6-45e5-474a-8d37-c0867ad9edb8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:44.905967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 342/679] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:44 2022] GET /identity/v3/role_assignments?user.id=de8484c3b114467ea59bebb2460ed546&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:44.906610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f32f6-45e5-474a-8d37-c0867ad9edb8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:44.924117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:44.977333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756a55e-302b-4cf1-a92d-335bf89f94fb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:45.018926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:45.030839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169866d-ec1a-4f26-8b35-9cf85d4d8c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:45.032402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169866d-ec1a-4f26-8b35-9cf85d4d8c00 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:45.034173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169866d-ec1a-4f26-8b35-9cf85d4d8c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:45.034173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169866d-ec1a-4f26-8b35-9cf85d4d8c00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:45.057604 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169866d-ec1a-4f26-8b35-9cf85d4d8c00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:45.060907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169866d-ec1a-4f26-8b35-9cf85d4d8c00 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-c169866d-ec1a-4f26-8b35-9cf85d4d8c00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:45.086844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:45.088001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:45.089319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:45.089319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:45.089319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:45.089319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:45.089319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 338/680] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:45.090791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169866d-ec1a-4f26-8b35-9cf85d4d8c00 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:45.109337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f32f6-45e5-474a-8d37-c0867ad9edb8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:45.171639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897c32f-2434-43c2-8d82-82abc100b3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:45.172916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897c32f-2434-43c2-8d82-82abc100b3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:45.172916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897c32f-2434-43c2-8d82-82abc100b3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:45.172916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897c32f-2434-43c2-8d82-82abc100b3a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:45.187781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897c32f-2434-43c2-8d82-82abc100b3a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:45.189593 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897c32f-2434-43c2-8d82-82abc100b3a7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a897c32f-2434-43c2-8d82-82abc100b3a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:45.217112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:45.218043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:45.218943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:45.218943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:45.218943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:45.218943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:45.218943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:45.218943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:45.218943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:45.218943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:45.218943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 343/681] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:45.219445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897c32f-2434-43c2-8d82-82abc100b3a7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:45.249570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169866d-ec1a-4f26-8b35-9cf85d4d8c00 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:45.298948 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66bf87-fcc5-4108-9f00-2f0cb02e5991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:45.299771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66bf87-fcc5-4108-9f00-2f0cb02e5991 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:45.300048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66bf87-fcc5-4108-9f00-2f0cb02e5991 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:45.300325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66bf87-fcc5-4108-9f00-2f0cb02e5991 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:45.322964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66bf87-fcc5-4108-9f00-2f0cb02e5991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:45.330684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8b66bf87-fcc5-4108-9f00-2f0cb02e5991 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:45.340612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:45.349874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 339/682] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:45.350775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66bf87-fcc5-4108-9f00-2f0cb02e5991 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:45.368268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897c32f-2434-43c2-8d82-82abc100b3a7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:45.412247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:45.426454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a989ad-4ecb-48ba-9e3f-70985f530b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:45.427376 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a989ad-4ecb-48ba-9e3f-70985f530b06 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:45.427665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a989ad-4ecb-48ba-9e3f-70985f530b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:45.427944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a989ad-4ecb-48ba-9e3f-70985f530b06 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:45.428681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a989ad-4ecb-48ba-9e3f-70985f530b06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:45.429014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a989ad-4ecb-48ba-9e3f-70985f530b06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:45.430412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a989ad-4ecb-48ba-9e3f-70985f530b06 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:45.450791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 344/683] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:45.451693 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a989ad-4ecb-48ba-9e3f-70985f530b06 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:45.499163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66bf87-fcc5-4108-9f00-2f0cb02e5991 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:45.543197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561f2ba-5c6b-4af1-ae8a-29ff93395b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:45.545033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561f2ba-5c6b-4af1-ae8a-29ff93395b48 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:45.545564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561f2ba-5c6b-4af1-ae8a-29ff93395b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:45.546190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561f2ba-5c6b-4af1-ae8a-29ff93395b48 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:45.547223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561f2ba-5c6b-4af1-ae8a-29ff93395b48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:45.547761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561f2ba-5c6b-4af1-ae8a-29ff93395b48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:45.549694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561f2ba-5c6b-4af1-ae8a-29ff93395b48 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:45.565035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 340/684] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:45.566443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561f2ba-5c6b-4af1-ae8a-29ff93395b48 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:45.638812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a989ad-4ecb-48ba-9e3f-70985f530b06 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:45.687830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:45.696771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746da0de-3a0a-4938-a252-f67afd7fd8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:45.697475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746da0de-3a0a-4938-a252-f67afd7fd8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:45.704228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746da0de-3a0a-4938-a252-f67afd7fd8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:45.704228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746da0de-3a0a-4938-a252-f67afd7fd8a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:45.704228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746da0de-3a0a-4938-a252-f67afd7fd8a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:45.704228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746da0de-3a0a-4938-a252-f67afd7fd8a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:45.704228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746da0de-3a0a-4938-a252-f67afd7fd8a5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:45.704600 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:45.705871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561f2ba-5c6b-4af1-ae8a-29ff93395b48 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:45.710253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 345/685] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:45.711110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746da0de-3a0a-4938-a252-f67afd7fd8a5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:45.774292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc0a22-244c-4e89-a4f1-99fbe5389340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:45.774292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0a22-244c-4e89-a4f1-99fbe5389340 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:45.774592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0a22-244c-4e89-a4f1-99fbe5389340 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:45.780901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0a22-244c-4e89-a4f1-99fbe5389340 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:45.786409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc0a22-244c-4e89-a4f1-99fbe5389340 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:45.788149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc0a22-244c-4e89-a4f1-99fbe5389340 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d1fc0a22-244c-4e89-a4f1-99fbe5389340 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:45.807361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:45.808381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:45.809402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:45.809402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:45.809402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:45.809402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:45.809402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:45.809402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:45.809402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:45.809402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:45.809402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:30:45.809402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:45.813323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 341/686] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:45.814536 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc0a22-244c-4e89-a4f1-99fbe5389340 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:45.888633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746da0de-3a0a-4938-a252-f67afd7fd8a5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:45.971879 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bb0f0-12d7-44f9-8f7f-281f70b672dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:45.972827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bb0f0-12d7-44f9-8f7f-281f70b672dd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:45.973327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bb0f0-12d7-44f9-8f7f-281f70b672dd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:45.976902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bb0f0-12d7-44f9-8f7f-281f70b672dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:45.976902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bb0f0-12d7-44f9-8f7f-281f70b672dd None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=809604baaed7486fb1020263e99e9c63) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:45.976902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bb0f0-12d7-44f9-8f7f-281f70b672dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:45.978263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bb0f0-12d7-44f9-8f7f-281f70b672dd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:45.995841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 346/687] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:45 2022] GET /identity/v3/role_assignments?user.id=809604baaed7486fb1020263e99e9c63&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:45.995841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bb0f0-12d7-44f9-8f7f-281f70b672dd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:46.017920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc0a22-244c-4e89-a4f1-99fbe5389340 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:46.088861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc366c2c-f89e-4578-87c1-d0a298b277af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:46.091033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc366c2c-f89e-4578-87c1-d0a298b277af None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:46.091033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc366c2c-f89e-4578-87c1-d0a298b277af None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:46.091033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc366c2c-f89e-4578-87c1-d0a298b277af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:46.117293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:46.118300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc366c2c-f89e-4578-87c1-d0a298b277af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:46.124221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc366c2c-f89e-4578-87c1-d0a298b277af None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-bc366c2c-f89e-4578-87c1-d0a298b277af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:46.170190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:46.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:46.182551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:46.182551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:46.182551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:46.182551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:46.182551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 342/688] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:46.189153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc366c2c-f89e-4578-87c1-d0a298b277af None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:46.217937 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:46.230447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bb0f0-12d7-44f9-8f7f-281f70b672dd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:46.307492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a325ed4-cc1b-489e-8f17-4a769d233e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:46.308305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a325ed4-cc1b-489e-8f17-4a769d233e99 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:46.308599 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a325ed4-cc1b-489e-8f17-4a769d233e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:46.308901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a325ed4-cc1b-489e-8f17-4a769d233e99 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:46.312643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a325ed4-cc1b-489e-8f17-4a769d233e99 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:46.312643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a325ed4-cc1b-489e-8f17-4a769d233e99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:46.314475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a325ed4-cc1b-489e-8f17-4a769d233e99 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:46.341201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 347/689] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Jul 18 16:30:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:46.347717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a325ed4-cc1b-489e-8f17-4a769d233e99 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:46.379720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc366c2c-f89e-4578-87c1-d0a298b277af None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:46.439593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdd81d-8e23-47d2-9a28-7b79abba5a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:46.440744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdd81d-8e23-47d2-9a28-7b79abba5a7d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:46.443966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdd81d-8e23-47d2-9a28-7b79abba5a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:46.444520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdd81d-8e23-47d2-9a28-7b79abba5a7d None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:46.445667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdd81d-8e23-47d2-9a28-7b79abba5a7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:46.446212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdd81d-8e23-47d2-9a28-7b79abba5a7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:46.447890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdd81d-8e23-47d2-9a28-7b79abba5a7d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:46.464437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 343/690] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:46.470125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdd81d-8e23-47d2-9a28-7b79abba5a7d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:46.489915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a325ed4-cc1b-489e-8f17-4a769d233e99 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:46.523857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:46.578342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849cb0a-9441-4fe0-8385-554bf02bf185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:46.581073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849cb0a-9441-4fe0-8385-554bf02bf185 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:46.581585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849cb0a-9441-4fe0-8385-554bf02bf185 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:46.583517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849cb0a-9441-4fe0-8385-554bf02bf185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:46.628597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:46.691096 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849cb0a-9441-4fe0-8385-554bf02bf185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:46.693205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849cb0a-9441-4fe0-8385-554bf02bf185 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:46.724264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdd81d-8e23-47d2-9a28-7b79abba5a7d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:46.818507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cded41-93cc-4712-a271-a1e73a43e11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:46.819510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cded41-93cc-4712-a271-a1e73a43e11f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:46.819697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cded41-93cc-4712-a271-a1e73a43e11f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:46.819977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cded41-93cc-4712-a271-a1e73a43e11f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:46.831425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 348/691] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:46.833267 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb3d67-cf55-42e6-9975-d6043a0c5e4f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:46.833788 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb3d67-cf55-42e6-9975-d6043a0c5e4f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:46.834308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb3d67-cf55-42e6-9975-d6043a0c5e4f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:46.836532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 349/692] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:46.837530 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb3d67-cf55-42e6-9975-d6043a0c5e4f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:46.843506 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cded41-93cc-4712-a271-a1e73a43e11f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:46.845024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cded41-93cc-4712-a271-a1e73a43e11f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:46.868319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-07cded41-93cc-4712-a271-a1e73a43e11f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:46.881624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:46.882714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:46.883689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:46.883689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:46.883689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:46.883689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:46.883689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:46.883689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:46.883689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:46.883689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:46.883689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:30:46.883689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:46.889244 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 344/693] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Jul 18 16:30:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:46.890622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf40439-dd72-49df-a7ef-233c8dc7da69 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:46.890999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf40439-dd72-49df-a7ef-233c8dc7da69 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:46.891189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf40439-dd72-49df-a7ef-233c8dc7da69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:46.940851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ebf40439-dd72-49df-a7ef-233c8dc7da69 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:46.945928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ebf40439-dd72-49df-a7ef-233c8dc7da69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:46.987794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb3d67-cf55-42e6-9975-d6043a0c5e4f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:47.061196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e744a1-e227-4475-a7c7-572819b56035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:47.062436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e744a1-e227-4475-a7c7-572819b56035 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:47.062926 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e744a1-e227-4475-a7c7-572819b56035 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:47.065887 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e744a1-e227-4475-a7c7-572819b56035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:47.089334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e744a1-e227-4475-a7c7-572819b56035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:47.091684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e744a1-e227-4475-a7c7-572819b56035 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:47.129004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 345/694] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:47.130293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf40439-dd72-49df-a7ef-233c8dc7da69 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-84e744a1-e227-4475-a7c7-572819b56035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:47.135480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:47.136370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:47.137292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:47.137292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:47.137292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:47.137292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:47.142367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 350/695] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:47.150117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e744a1-e227-4475-a7c7-572819b56035 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:47.241285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:47.265626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:47.303687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf40439-dd72-49df-a7ef-233c8dc7da69 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:47.305561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e744a1-e227-4475-a7c7-572819b56035 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:47.347912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d90623-b52e-4a64-8c74-36ac014f0752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:47.348628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d90623-b52e-4a64-8c74-36ac014f0752 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:47.348935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d90623-b52e-4a64-8c74-36ac014f0752 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:47.349156 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a7514-4633-48da-8716-1bc3226983a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:47.349296 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d90623-b52e-4a64-8c74-36ac014f0752 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:47.349939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d90623-b52e-4a64-8c74-36ac014f0752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:47.350245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d90623-b52e-4a64-8c74-36ac014f0752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:47.350613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a7514-4633-48da-8716-1bc3226983a3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:47.351097 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a7514-4633-48da-8716-1bc3226983a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:47.351571 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a7514-4633-48da-8716-1bc3226983a3 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:47.351694 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d90623-b52e-4a64-8c74-36ac014f0752 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:47.352724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a7514-4633-48da-8716-1bc3226983a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:47.353440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a7514-4633-48da-8716-1bc3226983a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:47.355592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a7514-4633-48da-8716-1bc3226983a3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:47.365651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 351/696] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:47.366521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d90623-b52e-4a64-8c74-36ac014f0752 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:47.385265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 346/697] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Jul 18 16:30:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:47.388348 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d74d4d-bd21-41e5-9185-a974ef0047f3 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:47.389016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d74d4d-bd21-41e5-9185-a974ef0047f3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:47.389913 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d74d4d-bd21-41e5-9185-a974ef0047f3 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:47.392269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 347/698] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:47.393763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d74d4d-bd21-41e5-9185-a974ef0047f3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:47.509438 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d90623-b52e-4a64-8c74-36ac014f0752 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:47.532056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d74d4d-bd21-41e5-9185-a974ef0047f3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:47.546018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:47.555418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:47.561496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a4e76-2da2-4e18-a579-f3d77410df3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:47.561496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a4e76-2da2-4e18-a579-f3d77410df3b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:47.561496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a4e76-2da2-4e18-a579-f3d77410df3b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:47.561496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a4e76-2da2-4e18-a579-f3d77410df3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:47.574601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d827a-b1a6-4375-9684-32a769b40f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:47.576116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d827a-b1a6-4375-9684-32a769b40f8d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:47.576557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d827a-b1a6-4375-9684-32a769b40f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:47.576915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a4e76-2da2-4e18-a579-f3d77410df3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:47.577031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d827a-b1a6-4375-9684-32a769b40f8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:47.579463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a4e76-2da2-4e18-a579-f3d77410df3b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:47.585975 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d827a-b1a6-4375-9684-32a769b40f8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:47.587594 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d827a-b1a6-4375-9684-32a769b40f8d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8e7d827a-b1a6-4375-9684-32a769b40f8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:47.603728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:47.604721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:47.605888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:47.605888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:47.605888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:47.605888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:47.605888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:47.605888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:47.605888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:47.605888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:47.605888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 348/699] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:47.607158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d827a-b1a6-4375-9684-32a769b40f8d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8f6a4e76-2da2-4e18-a579-f3d77410df3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:47.610062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:47.610940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:47.611818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:47.611818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:47.611818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:47.611818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:47.614546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 352/700] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:47.616199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c20a63-8b3d-464f-9c88-489cc95085b2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:47.617572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c20a63-8b3d-464f-9c88-489cc95085b2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:47.618076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c20a63-8b3d-464f-9c88-489cc95085b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:47.639671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-83c20a63-8b3d-464f-9c88-489cc95085b2 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:47.641240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-83c20a63-8b3d-464f-9c88-489cc95085b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:47.723799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 353/701] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:47.725102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c20a63-8b3d-464f-9c88-489cc95085b2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:47.746844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d827a-b1a6-4375-9684-32a769b40f8d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:47.808302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613094b1-0a1a-4b33-883f-7eea8a1ec322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:47.808915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613094b1-0a1a-4b33-883f-7eea8a1ec322 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:47.809752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613094b1-0a1a-4b33-883f-7eea8a1ec322 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:47.810048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613094b1-0a1a-4b33-883f-7eea8a1ec322 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:47.877787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c20a63-8b3d-464f-9c88-489cc95085b2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:47.882521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:47.890254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:47.916823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613094b1-0a1a-4b33-883f-7eea8a1ec322 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:47.919341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613094b1-0a1a-4b33-883f-7eea8a1ec322 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:47.922477 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d7537-36c4-4a61-86f5-31939a6284cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:47.926281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d7537-36c4-4a61-86f5-31939a6284cc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:47.926281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d7537-36c4-4a61-86f5-31939a6284cc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:47.926281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d7537-36c4-4a61-86f5-31939a6284cc None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:47.926281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d7537-36c4-4a61-86f5-31939a6284cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:47.926281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d7537-36c4-4a61-86f5-31939a6284cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:47.926641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d7537-36c4-4a61-86f5-31939a6284cc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:47.951901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 354/702] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:47.951901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d7537-36c4-4a61-86f5-31939a6284cc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:48.088068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 349/703] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 16:30:48.088912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613094b1-0a1a-4b33-883f-7eea8a1ec322 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:48.096640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d7537-36c4-4a61-86f5-31939a6284cc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:48.145940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf8410-5827-4c4f-b9ee-c843efcecb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:48.146610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf8410-5827-4c4f-b9ee-c843efcecb29 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:48.146920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf8410-5827-4c4f-b9ee-c843efcecb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:48.147173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf8410-5827-4c4f-b9ee-c843efcecb29 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:48.147905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf8410-5827-4c4f-b9ee-c843efcecb29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:48.148214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf8410-5827-4c4f-b9ee-c843efcecb29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:48.149650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf8410-5827-4c4f-b9ee-c843efcecb29 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:48.162512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 355/704] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:48.163359 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf8410-5827-4c4f-b9ee-c843efcecb29 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:48.201136 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:48.238334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:48.263492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613094b1-0a1a-4b33-883f-7eea8a1ec322 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:48.316414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd196b0d-8e1b-4ff7-8135-63c9262dd391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:48.317657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd196b0d-8e1b-4ff7-8135-63c9262dd391 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:48.318318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd196b0d-8e1b-4ff7-8135-63c9262dd391 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:48.318912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd196b0d-8e1b-4ff7-8135-63c9262dd391 None admin] PATH_INFO: `/v3/services/network` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:48.319917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd196b0d-8e1b-4ff7-8135-63c9262dd391 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:48.321631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf8410-5827-4c4f-b9ee-c843efcecb29 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:48.322162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd196b0d-8e1b-4ff7-8135-63c9262dd391 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-dd196b0d-8e1b-4ff7-8135-63c9262dd391 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:48.336221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:48.337293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:30:48.338157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:48.342491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 350/705] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:48 2022] GET /identity/v3/services/network => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:48.343329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd196b0d-8e1b-4ff7-8135-63c9262dd391 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:48.400008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304c38e-8e99-48ab-9064-2efcea468558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:48.400755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304c38e-8e99-48ab-9064-2efcea468558 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:48.401133 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304c38e-8e99-48ab-9064-2efcea468558 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:48.401413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304c38e-8e99-48ab-9064-2efcea468558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:48.402246 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304c38e-8e99-48ab-9064-2efcea468558 None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=bf341f1bb8944a3e9331678100b941a1) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:48.402544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304c38e-8e99-48ab-9064-2efcea468558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:48.404638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304c38e-8e99-48ab-9064-2efcea468558 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:48.410762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 356/706] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:48 2022] GET /identity/v3/role_assignments?user.id=bf341f1bb8944a3e9331678100b941a1&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:48.411860 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304c38e-8e99-48ab-9064-2efcea468558 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:48.484454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd196b0d-8e1b-4ff7-8135-63c9262dd391 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:48.538376 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:48.548585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304c38e-8e99-48ab-9064-2efcea468558 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:48.553108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a408-99c8-404f-a9c4-e9c37fd599ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:48.553108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a408-99c8-404f-a9c4-e9c37fd599ac None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:48.553108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a408-99c8-404f-a9c4-e9c37fd599ac None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:48.553108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a408-99c8-404f-a9c4-e9c37fd599ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:48.564693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894a408-99c8-404f-a9c4-e9c37fd599ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:48.567742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894a408-99c8-404f-a9c4-e9c37fd599ac None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:48.570325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-7894a408-99c8-404f-a9c4-e9c37fd599ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:48.589731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:48.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:48.594796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:48.594796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:48.594796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:48.594796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:48.594796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 351/707] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:48.594796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a408-99c8-404f-a9c4-e9c37fd599ac None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:48.625694 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdf21d-1e37-41f9-96fd-f4556be1a55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:48.626381 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdf21d-1e37-41f9-96fd-f4556be1a55b None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:48.626674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdf21d-1e37-41f9-96fd-f4556be1a55b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:48.626954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdf21d-1e37-41f9-96fd-f4556be1a55b None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:48.627724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afdf21d-1e37-41f9-96fd-f4556be1a55b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:48.629231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afdf21d-1e37-41f9-96fd-f4556be1a55b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:48.708908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 357/708] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:30:48 2022] POST /identity/v3/users => generated 254 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:30:48.708908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f1656-a3bb-4524-be5d-e4f5f2af076a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:48.709229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a408-99c8-404f-a9c4-e9c37fd599ac None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:48.709322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f1656-a3bb-4524-be5d-e4f5f2af076a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:48.709322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f1656-a3bb-4524-be5d-e4f5f2af076a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:48.709322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 358/709] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:48.713428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f1656-a3bb-4524-be5d-e4f5f2af076a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:48.755866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff8677-f558-4238-ad95-cd8021069795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:48.756801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff8677-f558-4238-ad95-cd8021069795 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:48.757704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff8677-f558-4238-ad95-cd8021069795 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:48.757995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff8677-f558-4238-ad95-cd8021069795 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:48.759151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff8677-f558-4238-ad95-cd8021069795 None admin] RBAC: Adding query filter params (name=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:48.759467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff8677-f558-4238-ad95-cd8021069795 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:48.762320 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff8677-f558-4238-ad95-cd8021069795 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:48.771170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 352/710] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:30:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:48.776006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779661f-df12-4849-a211-da61eb368cf0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:48.776304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779661f-df12-4849-a211-da61eb368cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:48.776656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779661f-df12-4849-a211-da61eb368cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:48.795523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4779661f-df12-4849-a211-da61eb368cf0 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:48.796764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4779661f-df12-4849-a211-da61eb368cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:48.859322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f1656-a3bb-4524-be5d-e4f5f2af076a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:48.861885 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:48.877662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 353/711] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:48.878531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779661f-df12-4849-a211-da61eb368cf0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:48.902249 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19d284-1540-4f0b-9202-cdb17808c4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:48.903651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19d284-1540-4f0b-9202-cdb17808c4fc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:48.903939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19d284-1540-4f0b-9202-cdb17808c4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:48.904268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19d284-1540-4f0b-9202-cdb17808c4fc None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:48.905060 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19d284-1540-4f0b-9202-cdb17808c4fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:48.905421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19d284-1540-4f0b-9202-cdb17808c4fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:48.906893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19d284-1540-4f0b-9202-cdb17808c4fc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:48.921769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 359/712] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:48.922683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19d284-1540-4f0b-9202-cdb17808c4fc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:48.944281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:49.010045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779661f-df12-4849-a211-da61eb368cf0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:49.055900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b80b58-2b97-4f98-9091-ec5b7fa03ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:49.056691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b80b58-2b97-4f98-9091-ec5b7fa03ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:49.057055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b80b58-2b97-4f98-9091-ec5b7fa03ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:49.057440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b80b58-2b97-4f98-9091-ec5b7fa03ca7 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:49.058162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b80b58-2b97-4f98-9091-ec5b7fa03ca7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:49.058473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b80b58-2b97-4f98-9091-ec5b7fa03ca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:49.060005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b80b58-2b97-4f98-9091-ec5b7fa03ca7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:49.063309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19d284-1540-4f0b-9202-cdb17808c4fc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:49.066813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 354/713] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:30:48 2022] GET /identity/v3/services?type=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:49.067647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b80b58-2b97-4f98-9091-ec5b7fa03ca7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:49.114067 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899e310-94a1-40d6-afe1-b6adff944b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:49.114881 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899e310-94a1-40d6-afe1-b6adff944b42 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:49.115170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899e310-94a1-40d6-afe1-b6adff944b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:49.120912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899e310-94a1-40d6-afe1-b6adff944b42 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:49.120912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899e310-94a1-40d6-afe1-b6adff944b42 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:49.120912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899e310-94a1-40d6-afe1-b6adff944b42 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8899e310-94a1-40d6-afe1-b6adff944b42 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:49.150155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:49.151161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:30:49.152062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:49.152827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 360/714] 10.222.0.88 () {56 vars in 1129 bytes} [Mon Jul 18 16:30:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:49.164544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899e310-94a1-40d6-afe1-b6adff944b42 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:49.189212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:49.247304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b80b58-2b97-4f98-9091-ec5b7fa03ca7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:49.296753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30715916-48ac-4e71-9267-c3d5b9041a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:49.297180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:49.298636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30715916-48ac-4e71-9267-c3d5b9041a96 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:49.298923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30715916-48ac-4e71-9267-c3d5b9041a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:49.299210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30715916-48ac-4e71-9267-c3d5b9041a96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:49.309936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30715916-48ac-4e71-9267-c3d5b9041a96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:49.311419 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30715916-48ac-4e71-9267-c3d5b9041a96 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:49.332648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899e310-94a1-40d6-afe1-b6adff944b42 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-30715916-48ac-4e71-9267-c3d5b9041a96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:49.340305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:49.342853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:49.344958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:49.344958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:49.344958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:49.344958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:49.344958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:49.344958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:49.344958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:49.344958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:49.346102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 355/715] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:49.354921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30715916-48ac-4e71-9267-c3d5b9041a96 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:49.374590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e49ec-192e-44e7-ad79-6e5abe03e2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:49.375186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e49ec-192e-44e7-ad79-6e5abe03e2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:49.375466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e49ec-192e-44e7-ad79-6e5abe03e2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:49.375740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e49ec-192e-44e7-ad79-6e5abe03e2e0 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:49.376454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e49ec-192e-44e7-ad79-6e5abe03e2e0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:49.376756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e49ec-192e-44e7-ad79-6e5abe03e2e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:49.378171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e49ec-192e-44e7-ad79-6e5abe03e2e0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:49.386205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 361/716] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Jul 18 16:30:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:49.392576 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e49ec-192e-44e7-ad79-6e5abe03e2e0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:49.542841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30715916-48ac-4e71-9267-c3d5b9041a96 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:49.572364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:49.592078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e49ec-192e-44e7-ad79-6e5abe03e2e0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:49.624074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc66630-f96b-4432-b291-a84774c5d452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:49.624074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc66630-f96b-4432-b291-a84774c5d452 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:49.624074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc66630-f96b-4432-b291-a84774c5d452 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:49.624074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc66630-f96b-4432-b291-a84774c5d452 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:49.624524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc66630-f96b-4432-b291-a84774c5d452 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:49.624524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc66630-f96b-4432-b291-a84774c5d452 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:49.627898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc66630-f96b-4432-b291-a84774c5d452 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:49.654686 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 356/717] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:49.661570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc66630-f96b-4432-b291-a84774c5d452 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:49.681031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32ac86-ec82-4ea9-9e6a-ed7ee4dde381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:49.681714 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32ac86-ec82-4ea9-9e6a-ed7ee4dde381 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:49.681995 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32ac86-ec82-4ea9-9e6a-ed7ee4dde381 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:49.682263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32ac86-ec82-4ea9-9e6a-ed7ee4dde381 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:49.682975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32ac86-ec82-4ea9-9e6a-ed7ee4dde381 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:49.683283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32ac86-ec82-4ea9-9e6a-ed7ee4dde381 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:49.690010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32ac86-ec82-4ea9-9e6a-ed7ee4dde381 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:49.697226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:49.706710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 362/718] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Jul 18 16:30:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:49.812938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc66630-f96b-4432-b291-a84774c5d452 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:49.865806 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb52df-bc22-429c-b4fb-35907cf074af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:49.866903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb52df-bc22-429c-b4fb-35907cf074af None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:49.866903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb52df-bc22-429c-b4fb-35907cf074af None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:49.867563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb52df-bc22-429c-b4fb-35907cf074af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:49.868533 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb52df-bc22-429c-b4fb-35907cf074af None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=69ee145599384c6da2cba85a42832dcd) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:49.868875 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb52df-bc22-429c-b4fb-35907cf074af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:49.870433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb52df-bc22-429c-b4fb-35907cf074af None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:49.886334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 357/719] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:49 2022] GET /identity/v3/role_assignments?user.id=69ee145599384c6da2cba85a42832dcd&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=81457691f3a2456fa33358cb6a922d05 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:50.902487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31da28-7640-400b-9675-6deb87f07f3e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:50.905790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31da28-7640-400b-9675-6deb87f07f3e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:50.905790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31da28-7640-400b-9675-6deb87f07f3e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:50.906297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 363/720] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:50.922705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01553a6-6965-43c0-97ac-347e09a18291 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:50.923152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01553a6-6965-43c0-97ac-347e09a18291 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:50.923359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01553a6-6965-43c0-97ac-347e09a18291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:50.946156 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b01553a6-6965-43c0-97ac-347e09a18291 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:50.947376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b01553a6-6965-43c0-97ac-347e09a18291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:50.988978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:51.044416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 358/721] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:51.084363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31da28-7640-400b-9675-6deb87f07f3e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:51.202809 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669a424-6d31-4f52-82f1-faec84293c00 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:51.203118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669a424-6d31-4f52-82f1-faec84293c00 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:51.203297 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669a424-6d31-4f52-82f1-faec84293c00 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:51.204993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 359/722] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:51.211286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa25d1-8e64-4849-8788-808d2ebaae21 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:51.211593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa25d1-8e64-4849-8788-808d2ebaae21 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:51.211859 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa25d1-8e64-4849-8788-808d2ebaae21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:51.214905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31da28-7640-400b-9675-6deb87f07f3e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:51.250012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f9fa25d1-8e64-4849-8788-808d2ebaae21 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:51.252039 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f9fa25d1-8e64-4849-8788-808d2ebaae21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:51.258963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d0f05-7a70-44f9-a785-a48195cc438b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:51.260040 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d0f05-7a70-44f9-a785-a48195cc438b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:51.260812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d0f05-7a70-44f9-a785-a48195cc438b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:51.261279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d0f05-7a70-44f9-a785-a48195cc438b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:51.281546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d0f05-7a70-44f9-a785-a48195cc438b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-877d0f05-7a70-44f9-a785-a48195cc438b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:51.300912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:51.301960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 364/723] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:51.302806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d0f05-7a70-44f9-a785-a48195cc438b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:51.384942 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 360/724] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 16:30:51.421272 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:51.432628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa25d1-8e64-4849-8788-808d2ebaae21 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:51.487940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d0f05-7a70-44f9-a785-a48195cc438b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:51.565400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1c991-fb03-4e35-a5d6-9ebc9d66bcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:51.566613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1c991-fb03-4e35-a5d6-9ebc9d66bcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:51.568223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1c991-fb03-4e35-a5d6-9ebc9d66bcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:51.568982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1c991-fb03-4e35-a5d6-9ebc9d66bcf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:51.570535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1c991-fb03-4e35-a5d6-9ebc9d66bcf9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:51.571238 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1c991-fb03-4e35-a5d6-9ebc9d66bcf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:51.572821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1c991-fb03-4e35-a5d6-9ebc9d66bcf9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:51.589590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 365/725] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:51.596289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1c991-fb03-4e35-a5d6-9ebc9d66bcf9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:51.620551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:51.630456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa25d1-8e64-4849-8788-808d2ebaae21 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:51.684821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979139f-dd50-4e20-9f97-a4200ec7d9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:51.686072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979139f-dd50-4e20-9f97-a4200ec7d9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:51.686371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979139f-dd50-4e20-9f97-a4200ec7d9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:51.686652 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979139f-dd50-4e20-9f97-a4200ec7d9f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:51.698366 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979139f-dd50-4e20-9f97-a4200ec7d9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5979139f-dd50-4e20-9f97-a4200ec7d9f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:51.706849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:51.707889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:51.708736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 361/726] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:51.714240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979139f-dd50-4e20-9f97-a4200ec7d9f0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:51.791185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1c991-fb03-4e35-a5d6-9ebc9d66bcf9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:51.818176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:51.830850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae96a70-59c2-48b5-814c-688c7d34a3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:51.831653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae96a70-59c2-48b5-814c-688c7d34a3cf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:51.831935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae96a70-59c2-48b5-814c-688c7d34a3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:51.832212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae96a70-59c2-48b5-814c-688c7d34a3cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:51.844625 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae96a70-59c2-48b5-814c-688c7d34a3cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:51.846487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae96a70-59c2-48b5-814c-688c7d34a3cf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:51.864252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979139f-dd50-4e20-9f97-a4200ec7d9f0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6ae96a70-59c2-48b5-814c-688c7d34a3cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:51.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:51.866614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:51.867759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:51.867759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:51.867759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:51.867759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:51.867987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 366/727] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:51.874233 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd6b64-f264-4fd0-a72c-17f62543c3e2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:51.874581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd6b64-f264-4fd0-a72c-17f62543c3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:51.874866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd6b64-f264-4fd0-a72c-17f62543c3e2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:51.876681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 367/728] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:51.877481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd6b64-f264-4fd0-a72c-17f62543c3e2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:51.916647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9656fa-6e37-4cd0-9fa5-7f0c8d095cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:51.917903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9656fa-6e37-4cd0-9fa5-7f0c8d095cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:51.918798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9656fa-6e37-4cd0-9fa5-7f0c8d095cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:51.919084 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9656fa-6e37-4cd0-9fa5-7f0c8d095cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:51.919847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9656fa-6e37-4cd0-9fa5-7f0c8d095cc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:51.925321 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9656fa-6e37-4cd0-9fa5-7f0c8d095cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:51.929323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9656fa-6e37-4cd0-9fa5-7f0c8d095cc7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:51.945762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 362/729] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:51.947093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fda0d-4356-4759-9ee7-6cfe6b032761 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:51.947389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fda0d-4356-4759-9ee7-6cfe6b032761 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:51.947879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fda0d-4356-4759-9ee7-6cfe6b032761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:51.967235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-747fda0d-4356-4759-9ee7-6cfe6b032761 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:51.968441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-747fda0d-4356-4759-9ee7-6cfe6b032761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:51.983190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:52.009853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd6b64-f264-4fd0-a72c-17f62543c3e2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:52.050963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f88bc-3558-458a-abbf-d7b7b49d5a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:52.051552 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f88bc-3558-458a-abbf-d7b7b49d5a01 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.051835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f88bc-3558-458a-abbf-d7b7b49d5a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.052114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f88bc-3558-458a-abbf-d7b7b49d5a01 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.052979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f88bc-3558-458a-abbf-d7b7b49d5a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:52.053291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f88bc-3558-458a-abbf-d7b7b49d5a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:52.054744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f88bc-3558-458a-abbf-d7b7b49d5a01 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:52.068283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 363/730] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:52.069613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fda0d-4356-4759-9ee7-6cfe6b032761 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:52.078754 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 368/731] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:52.082949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f88bc-3558-458a-abbf-d7b7b49d5a01 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:52.131131 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:52.206934 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fda0d-4356-4759-9ee7-6cfe6b032761 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:52.214094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f88bc-3558-458a-abbf-d7b7b49d5a01 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:52.247082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6d2f2-55b1-4c30-a93d-b089939b7572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:52.248013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6d2f2-55b1-4c30-a93d-b089939b7572 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.248298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6d2f2-55b1-4c30-a93d-b089939b7572 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.248577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6d2f2-55b1-4c30-a93d-b089939b7572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.260246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6d2f2-55b1-4c30-a93d-b089939b7572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:52.261722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6d2f2-55b1-4c30-a93d-b089939b7572 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:52.263046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4570d6-f7f5-465a-8eb3-52e5ddd3f99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:52.264240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4570d6-f7f5-465a-8eb3-52e5ddd3f99a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.264745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4570d6-f7f5-465a-8eb3-52e5ddd3f99a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.265544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4570d6-f7f5-465a-8eb3-52e5ddd3f99a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.271604 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:52.274908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4570d6-f7f5-465a-8eb3-52e5ddd3f99a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:52.276712 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4570d6-f7f5-465a-8eb3-52e5ddd3f99a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-06c6d2f2-55b1-4c30-a93d-b089939b7572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:52.282557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:52.286522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:52.287556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:52.287556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:52.287556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:52.287556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:52.287556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 364/732] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:52.287556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6d2f2-55b1-4c30-a93d-b089939b7572 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8e4570d6-f7f5-465a-8eb3-52e5ddd3f99a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:52.298689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:52.299655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:52.300608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:52.300608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:52.300608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:52.300608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:52.300608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:52.300608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:52.300608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:52.300608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:52.300608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:30:52.300608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:52.300608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 369/733] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:52.301986 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284e5ff-ddcb-42be-9270-13155e7efbb1 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.302208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284e5ff-ddcb-42be-9270-13155e7efbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.302490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284e5ff-ddcb-42be-9270-13155e7efbb1 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.303875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 370/734] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:52.307983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284e5ff-ddcb-42be-9270-13155e7efbb1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:52.398963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6d2f2-55b1-4c30-a93d-b089939b7572 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:52.420586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:52.430744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284e5ff-ddcb-42be-9270-13155e7efbb1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:52.435003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1dd7b-a012-4137-8927-675647f097ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:52.435677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1dd7b-a012-4137-8927-675647f097ed None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.435965 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1dd7b-a012-4137-8927-675647f097ed None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.436242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1dd7b-a012-4137-8927-675647f097ed None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.436995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1dd7b-a012-4137-8927-675647f097ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:52.438372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1dd7b-a012-4137-8927-675647f097ed None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:52.466796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd86295-bd4a-4edf-91ac-d2efb044e3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:52.467464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd86295-bd4a-4edf-91ac-d2efb044e3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.467744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd86295-bd4a-4edf-91ac-d2efb044e3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.468016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd86295-bd4a-4edf-91ac-d2efb044e3e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.468818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd86295-bd4a-4edf-91ac-d2efb044e3e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:52.469166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd86295-bd4a-4edf-91ac-d2efb044e3e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:52.470536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd86295-bd4a-4edf-91ac-d2efb044e3e7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:52.482532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 371/735] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:52.483317 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd86295-bd4a-4edf-91ac-d2efb044e3e7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:52.539745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 365/736] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:30:52 2022] POST /identity/v3/services => generated 240 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:30:52.541549 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8943900-3b3b-4e7a-a185-acc68e5c37ee None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.541847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8943900-3b3b-4e7a-a185-acc68e5c37ee None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.542176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8943900-3b3b-4e7a-a185-acc68e5c37ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.566381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f8943900-3b3b-4e7a-a185-acc68e5c37ee None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:52.567569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f8943900-3b3b-4e7a-a185-acc68e5c37ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:52.592237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd86295-bd4a-4edf-91ac-d2efb044e3e7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:52.636142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:52.647595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eda1f0-5009-4c54-9b6f-8cb8b46dd6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:52.648283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eda1f0-5009-4c54-9b6f-8cb8b46dd6aa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.648785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 366/737] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:52 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:52.650313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e476198-c449-4911-b47a-3f1480c7a44a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.650613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e476198-c449-4911-b47a-3f1480c7a44a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.650892 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e476198-c449-4911-b47a-3f1480c7a44a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.657141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 367/738] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:52.657141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e476198-c449-4911-b47a-3f1480c7a44a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:52.657428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eda1f0-5009-4c54-9b6f-8cb8b46dd6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.657681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eda1f0-5009-4c54-9b6f-8cb8b46dd6aa None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.658662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eda1f0-5009-4c54-9b6f-8cb8b46dd6aa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:52.658983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eda1f0-5009-4c54-9b6f-8cb8b46dd6aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:52.662086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eda1f0-5009-4c54-9b6f-8cb8b46dd6aa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:52.687534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 372/739] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Jul 18 16:30:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:52.688848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12204d37-2e51-455d-85f0-db6e9be012ff None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.689170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12204d37-2e51-455d-85f0-db6e9be012ff None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.689445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12204d37-2e51-455d-85f0-db6e9be012ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.707484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-12204d37-2e51-455d-85f0-db6e9be012ff None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:52.708899 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-12204d37-2e51-455d-85f0-db6e9be012ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:52.711735 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:52.779596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e476198-c449-4911-b47a-3f1480c7a44a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:52.788928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 373/740] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:52 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 16:30:52.788928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12204d37-2e51-455d-85f0-db6e9be012ff None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:52.819812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef294f76-4653-4e63-98ff-4e993dcf4c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:52.820548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef294f76-4653-4e63-98ff-4e993dcf4c13 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.820835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef294f76-4653-4e63-98ff-4e993dcf4c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.821144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef294f76-4653-4e63-98ff-4e993dcf4c13 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.829762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef294f76-4653-4e63-98ff-4e993dcf4c13 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:52.831302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef294f76-4653-4e63-98ff-4e993dcf4c13 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ef294f76-4653-4e63-98ff-4e993dcf4c13 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:52.848751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:52.849767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:52.850725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:52.850725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:52.850725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:52.850725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:52.850725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:52.850725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:52.850725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:52.850725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:52.850725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:30:52.850725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:52.850725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 368/741] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Jul 18 16:30:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:52.851304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef294f76-4653-4e63-98ff-4e993dcf4c13 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:52.906324 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12204d37-2e51-455d-85f0-db6e9be012ff None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:52.932550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:52.951243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a6a4a-e7fd-4099-8ff5-950e2ad5b607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:52.951832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a6a4a-e7fd-4099-8ff5-950e2ad5b607 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:52.952112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a6a4a-e7fd-4099-8ff5-950e2ad5b607 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:52.952384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a6a4a-e7fd-4099-8ff5-950e2ad5b607 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:52.953139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a6a4a-e7fd-4099-8ff5-950e2ad5b607 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:52.954618 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a6a4a-e7fd-4099-8ff5-950e2ad5b607 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:52.986998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef294f76-4653-4e63-98ff-4e993dcf4c13 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:53.014853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:53.024289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 374/742] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:30:52 2022] POST /identity/v3/services => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:30:53.024289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a6a4a-e7fd-4099-8ff5-950e2ad5b607 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:53.040922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a099cde-c40b-4886-a861-8f2b42ca992d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:53.040922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a099cde-c40b-4886-a861-8f2b42ca992d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:53.040922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a099cde-c40b-4886-a861-8f2b42ca992d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:53.040922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a099cde-c40b-4886-a861-8f2b42ca992d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:53.145601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a099cde-c40b-4886-a861-8f2b42ca992d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:53.148081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a099cde-c40b-4886-a861-8f2b42ca992d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:53.181676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a6a4a-e7fd-4099-8ff5-950e2ad5b607 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:53.206768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:53.227816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3bf44-0d91-4f8a-b9d5-e84cb4268651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:53.228618 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3bf44-0d91-4f8a-b9d5-e84cb4268651 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:53.228919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3bf44-0d91-4f8a-b9d5-e84cb4268651 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:53.229198 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3bf44-0d91-4f8a-b9d5-e84cb4268651 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:53.246368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3bf44-0d91-4f8a-b9d5-e84cb4268651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f5a3bf44-0d91-4f8a-b9d5-e84cb4268651 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:53.254707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:53.255674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:53.256514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 375/743] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:53.257297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3bf44-0d91-4f8a-b9d5-e84cb4268651 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:53.292111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 369/744] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:30:53.293328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a099cde-c40b-4886-a861-8f2b42ca992d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:53.329525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:53.380015 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3bf44-0d91-4f8a-b9d5-e84cb4268651 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:53.418638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeede0-b590-4d80-ad6d-ac057a765e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:53.419276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeede0-b590-4d80-ad6d-ac057a765e15 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:53.419561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeede0-b590-4d80-ad6d-ac057a765e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:53.419841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eeede0-b590-4d80-ad6d-ac057a765e15 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:53.420703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eeede0-b590-4d80-ad6d-ac057a765e15 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:53.421066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eeede0-b590-4d80-ad6d-ac057a765e15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:53.422555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eeede0-b590-4d80-ad6d-ac057a765e15 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:53.438013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a099cde-c40b-4886-a861-8f2b42ca992d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:53.448127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 376/745] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Jul 18 16:30:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:53.448127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeede0-b590-4d80-ad6d-ac057a765e15 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:53.481793 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128442e-bbc8-4b5d-8b41-e2fdc4a08113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:53.482463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128442e-bbc8-4b5d-8b41-e2fdc4a08113 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:53.482747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128442e-bbc8-4b5d-8b41-e2fdc4a08113 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:53.483022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7128442e-bbc8-4b5d-8b41-e2fdc4a08113 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:53.483830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128442e-bbc8-4b5d-8b41-e2fdc4a08113 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:53.484133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128442e-bbc8-4b5d-8b41-e2fdc4a08113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:53.485657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7128442e-bbc8-4b5d-8b41-e2fdc4a08113 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:53.494938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 370/746] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:53.495638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128442e-bbc8-4b5d-8b41-e2fdc4a08113 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:53.507402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:53.576461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eeede0-b590-4d80-ad6d-ac057a765e15 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:53.596072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:53.616614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c2ede-f44c-4a9f-a00f-60b97ea4e92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:53.618069 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c2ede-f44c-4a9f-a00f-60b97ea4e92b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:53.618590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c2ede-f44c-4a9f-a00f-60b97ea4e92b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:53.619074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c2ede-f44c-4a9f-a00f-60b97ea4e92b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:53.630041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7128442e-bbc8-4b5d-8b41-e2fdc4a08113 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:53.631190 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c2ede-f44c-4a9f-a00f-60b97ea4e92b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:53.632929 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5c2ede-f44c-4a9f-a00f-60b97ea4e92b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-aa5c2ede-f44c-4a9f-a00f-60b97ea4e92b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:53.654075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:53.655036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:53.655937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:53.655937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:53.655937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:53.655937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:53.658395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 377/747] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:53.659433 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c2ede-f44c-4a9f-a00f-60b97ea4e92b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:53.677578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609cef7-7d20-4959-8860-bdfe3eb82377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:53.678168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609cef7-7d20-4959-8860-bdfe3eb82377 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:53.678454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609cef7-7d20-4959-8860-bdfe3eb82377 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:53.678736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609cef7-7d20-4959-8860-bdfe3eb82377 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:53.777572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609cef7-7d20-4959-8860-bdfe3eb82377 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:53.779993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609cef7-7d20-4959-8860-bdfe3eb82377 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:53.780893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5c2ede-f44c-4a9f-a00f-60b97ea4e92b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:53.791951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:53.830359 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aced89-5618-4e88-982b-afb278432a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:53.831209 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aced89-5618-4e88-982b-afb278432a71 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:53.831498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aced89-5618-4e88-982b-afb278432a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:53.831781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aced89-5618-4e88-982b-afb278432a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:53.848417 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aced89-5618-4e88-982b-afb278432a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:53.850635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aced89-5618-4e88-982b-afb278432a71 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-16aced89-5618-4e88-982b-afb278432a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:53.873940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:53.874928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:53.875828 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:53.875828 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:53.875828 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:53.875828 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:53.875828 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 378/748] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:53.876605 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aced89-5618-4e88-982b-afb278432a71 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:53.937182 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:54.002506 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aced89-5618-4e88-982b-afb278432a71 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:54.032309 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 371/749] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:30:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:30:54.032309 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609cef7-7d20-4959-8860-bdfe3eb82377 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:54.050993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29011a-90b8-40be-a93e-0cc42c52be7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:54.052504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29011a-90b8-40be-a93e-0cc42c52be7f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.053344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29011a-90b8-40be-a93e-0cc42c52be7f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.053832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29011a-90b8-40be-a93e-0cc42c52be7f None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.054787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29011a-90b8-40be-a93e-0cc42c52be7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:54.055309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29011a-90b8-40be-a93e-0cc42c52be7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:54.057678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d29011a-90b8-40be-a93e-0cc42c52be7f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:54.077077 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 379/750] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:54.078969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29011a-90b8-40be-a93e-0cc42c52be7f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:54.161280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609cef7-7d20-4959-8860-bdfe3eb82377 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:54.174278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:54.178164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:54.200489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ba4a-86c9-4a29-97b0-2101b353768e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:54.201310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ba4a-86c9-4a29-97b0-2101b353768e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.201692 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ba4a-86c9-4a29-97b0-2101b353768e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.202069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ba4a-86c9-4a29-97b0-2101b353768e None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.202963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6ba4a-86c9-4a29-97b0-2101b353768e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:54.203358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6ba4a-86c9-4a29-97b0-2101b353768e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:54.204654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29011a-90b8-40be-a93e-0cc42c52be7f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:54.205019 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6ba4a-86c9-4a29-97b0-2101b353768e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:54.221541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 372/751] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:54.222623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ba4a-86c9-4a29-97b0-2101b353768e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:54.250136 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79fa13-6871-4a47-8195-d89f8823420f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:54.251728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79fa13-6871-4a47-8195-d89f8823420f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.252171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79fa13-6871-4a47-8195-d89f8823420f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.252546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79fa13-6871-4a47-8195-d89f8823420f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.268658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79fa13-6871-4a47-8195-d89f8823420f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:54.270521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79fa13-6871-4a47-8195-d89f8823420f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bd79fa13-6871-4a47-8195-d89f8823420f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:54.296921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:54.298098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:54.299212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:54.299212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:54.299212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:54.299212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:54.301309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 380/752] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:54.302527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79fa13-6871-4a47-8195-d89f8823420f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:54.346857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ba4a-86c9-4a29-97b0-2101b353768e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:54.385099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1df3e-bd0b-4df1-8cc0-07a7e33851cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:54.386150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1df3e-bd0b-4df1-8cc0-07a7e33851cd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.386626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1df3e-bd0b-4df1-8cc0-07a7e33851cd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.387095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1df3e-bd0b-4df1-8cc0-07a7e33851cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.395326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1df3e-bd0b-4df1-8cc0-07a7e33851cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:54.396981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1df3e-bd0b-4df1-8cc0-07a7e33851cd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-83d1df3e-bd0b-4df1-8cc0-07a7e33851cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:54.416683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:54.417826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:54.418754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:54.418754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:54.418754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:54.418754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:54.418754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:54.418754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:54.418754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:54.418754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:54.425006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 373/753] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:54.426194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1df3e-bd0b-4df1-8cc0-07a7e33851cd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:54.443527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79fa13-6871-4a47-8195-d89f8823420f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:54.460594 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:54.475228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:54.480393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72872380-6585-43fe-ad8a-a5bb36714589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:54.481307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872380-6585-43fe-ad8a-a5bb36714589 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.481596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872380-6585-43fe-ad8a-a5bb36714589 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.481876 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72872380-6585-43fe-ad8a-a5bb36714589 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.489996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72872380-6585-43fe-ad8a-a5bb36714589 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:54.491469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72872380-6585-43fe-ad8a-a5bb36714589 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-72872380-6585-43fe-ad8a-a5bb36714589 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:54.507521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:54.508527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 381/754] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Jul 18 16:30:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:54.510170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72872380-6585-43fe-ad8a-a5bb36714589 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:54.549085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1df3e-bd0b-4df1-8cc0-07a7e33851cd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:54.588187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6b446-2d5a-432d-b64c-dddabb7bfdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:54.589401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6b446-2d5a-432d-b64c-dddabb7bfdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.589897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6b446-2d5a-432d-b64c-dddabb7bfdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.590453 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6b446-2d5a-432d-b64c-dddabb7bfdd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.591397 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6b446-2d5a-432d-b64c-dddabb7bfdd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:54.591908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6b446-2d5a-432d-b64c-dddabb7bfdd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:54.593675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6b446-2d5a-432d-b64c-dddabb7bfdd4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:54.607184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 374/755] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:54.608455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6b446-2d5a-432d-b64c-dddabb7bfdd4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:54.619392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72872380-6585-43fe-ad8a-a5bb36714589 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:54.658204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252da70c-e956-48e4-ac87-7fa237f14ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:54.658882 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da70c-e956-48e4-ac87-7fa237f14ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.659164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da70c-e956-48e4-ac87-7fa237f14ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.659495 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252da70c-e956-48e4-ac87-7fa237f14ac6 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.660457 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252da70c-e956-48e4-ac87-7fa237f14ac6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:54.660774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252da70c-e956-48e4-ac87-7fa237f14ac6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:54.662362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252da70c-e956-48e4-ac87-7fa237f14ac6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:54.676642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 382/756] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:54.677592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252da70c-e956-48e4-ac87-7fa237f14ac6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:54.713357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:54.732396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6b446-2d5a-432d-b64c-dddabb7bfdd4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:54.736071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:54.771368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11532c92-f8c4-48fe-bd3f-1f8fe10c7821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:54.772089 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11532c92-f8c4-48fe-bd3f-1f8fe10c7821 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.772371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11532c92-f8c4-48fe-bd3f-1f8fe10c7821 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.772649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11532c92-f8c4-48fe-bd3f-1f8fe10c7821 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.773529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11532c92-f8c4-48fe-bd3f-1f8fe10c7821 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:54.773853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11532c92-f8c4-48fe-bd3f-1f8fe10c7821 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:54.775282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11532c92-f8c4-48fe-bd3f-1f8fe10c7821 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:54.787330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252da70c-e956-48e4-ac87-7fa237f14ac6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:54.793246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 375/757] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Jul 18 16:30:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:54.795341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba31d9-9494-4447-a178-89462748d1d8 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.795631 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba31d9-9494-4447-a178-89462748d1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.795908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba31d9-9494-4447-a178-89462748d1d8 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.800795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 376/758] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:54.800795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba31d9-9494-4447-a178-89462748d1d8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:54.831722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f49a4-36b5-4dbd-80f3-1826105f6085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:54.832869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f49a4-36b5-4dbd-80f3-1826105f6085 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.833405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f49a4-36b5-4dbd-80f3-1826105f6085 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.836203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f49a4-36b5-4dbd-80f3-1826105f6085 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.842200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f49a4-36b5-4dbd-80f3-1826105f6085 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:54.843890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f49a4-36b5-4dbd-80f3-1826105f6085 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-792f49a4-36b5-4dbd-80f3-1826105f6085 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:54.859041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:54.859984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:54.860863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:54.860863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:54.860863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:54.860863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:54.860863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:54.860863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:54.860863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:54.860863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:54.862996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 383/759] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:54.863961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f49a4-36b5-4dbd-80f3-1826105f6085 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:54.912665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba31d9-9494-4447-a178-89462748d1d8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:54.955437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764688fd-3a73-482e-8ffb-09020e6376ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:54.956095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764688fd-3a73-482e-8ffb-09020e6376ed None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.956380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764688fd-3a73-482e-8ffb-09020e6376ed None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.956915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764688fd-3a73-482e-8ffb-09020e6376ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.957524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764688fd-3a73-482e-8ffb-09020e6376ed None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=bf341f1bb8944a3e9331678100b941a1) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:54.957832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764688fd-3a73-482e-8ffb-09020e6376ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:54.959263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764688fd-3a73-482e-8ffb-09020e6376ed None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:54.965374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 377/760] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:54 2022] GET /identity/v3/role_assignments?user.id=bf341f1bb8944a3e9331678100b941a1&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=81457691f3a2456fa33358cb6a922d05 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:54.970400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9815276-5334-4bab-95d1-8639b712603d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:54.970710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9815276-5334-4bab-95d1-8639b712603d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:54.970965 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9815276-5334-4bab-95d1-8639b712603d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:54.989429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f49a4-36b5-4dbd-80f3-1826105f6085 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:54.990969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f9815276-5334-4bab-95d1-8639b712603d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:54.992510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f9815276-5334-4bab-95d1-8639b712603d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:54.995897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:55.013947 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:55.042285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd4276-3363-493c-af6b-7f29a80f6f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:55.044026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4276-3363-493c-af6b-7f29a80f6f94 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.044540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4276-3363-493c-af6b-7f29a80f6f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.045086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4276-3363-493c-af6b-7f29a80f6f94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.058064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd4276-3363-493c-af6b-7f29a80f6f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:55.061078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd4276-3363-493c-af6b-7f29a80f6f94 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4ccd4276-3363-493c-af6b-7f29a80f6f94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:55.080480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:55.081481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:55.082355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:55.082355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:55.082355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:55.082355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:55.082586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 378/761] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:30:55.082586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9815276-5334-4bab-95d1-8639b712603d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:55.089334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 384/762] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:55.091397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd4276-3363-493c-af6b-7f29a80f6f94 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:55.215008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9815276-5334-4bab-95d1-8639b712603d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:55.223092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd4276-3363-493c-af6b-7f29a80f6f94 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:55.254167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38493bcd-de77-4cc9-acc3-88cc5ee4dac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:55.254841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38493bcd-de77-4cc9-acc3-88cc5ee4dac9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.255368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38493bcd-de77-4cc9-acc3-88cc5ee4dac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.255368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38493bcd-de77-4cc9-acc3-88cc5ee4dac9 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.256174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38493bcd-de77-4cc9-acc3-88cc5ee4dac9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:55.256483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38493bcd-de77-4cc9-acc3-88cc5ee4dac9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:55.257963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38493bcd-de77-4cc9-acc3-88cc5ee4dac9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:55.264371 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534f0f8-ddb3-419d-990f-55442722f573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:55.265071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534f0f8-ddb3-419d-990f-55442722f573 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.265358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534f0f8-ddb3-419d-990f-55442722f573 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.265635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534f0f8-ddb3-419d-990f-55442722f573 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.266370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534f0f8-ddb3-419d-990f-55442722f573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:55.266675 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534f0f8-ddb3-419d-990f-55442722f573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:55.268048 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534f0f8-ddb3-419d-990f-55442722f573 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:55.268747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:55.273773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 379/763] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:55.274992 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38493bcd-de77-4cc9-acc3-88cc5ee4dac9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:55.282004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 385/764] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:55.283333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3dfba-8f80-4e1e-bef7-ee26948a357a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.283627 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3dfba-8f80-4e1e-bef7-ee26948a357a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.283902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3dfba-8f80-4e1e-bef7-ee26948a357a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.285412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 386/765] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:55.286123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3dfba-8f80-4e1e-bef7-ee26948a357a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:55.317372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:55.400416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38493bcd-de77-4cc9-acc3-88cc5ee4dac9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:55.411117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3dfba-8f80-4e1e-bef7-ee26948a357a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:55.438093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6a364-e270-4682-8d6b-6e564dc36587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:55.439398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6a364-e270-4682-8d6b-6e564dc36587 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.439923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6a364-e270-4682-8d6b-6e564dc36587 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.440408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6a364-e270-4682-8d6b-6e564dc36587 None admin] PATH_INFO: `/v3/services/network` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.441284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6a364-e270-4682-8d6b-6e564dc36587 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:55.443145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6a364-e270-4682-8d6b-6e564dc36587 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:55.447130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680eccb-427d-4b41-bd7c-687e5e804e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:55.447783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680eccb-427d-4b41-bd7c-687e5e804e5a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.448069 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680eccb-427d-4b41-bd7c-687e5e804e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.448349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680eccb-427d-4b41-bd7c-687e5e804e5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.449211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680eccb-427d-4b41-bd7c-687e5e804e5a None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=3f38abef63e24567a995b30c144177fa) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:55.449523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680eccb-427d-4b41-bd7c-687e5e804e5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:55.451012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680eccb-427d-4b41-bd7c-687e5e804e5a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:55.456641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 387/766] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:30:55 2022] GET /identity/v3/role_assignments?user.id=3f38abef63e24567a995b30c144177fa&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:55.457484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680eccb-427d-4b41-bd7c-687e5e804e5a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5ef6a364-e270-4682-8d6b-6e564dc36587 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:55.457597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:55.458860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:30:55.459923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:55.462432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 380/767] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:55 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:55.463897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7fd52-327e-478b-b1ef-3d9035f7c7de None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.464194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7fd52-327e-478b-b1ef-3d9035f7c7de None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.464477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7fd52-327e-478b-b1ef-3d9035f7c7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.486147 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-59e7fd52-327e-478b-b1ef-3d9035f7c7de None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:55.487308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-59e7fd52-327e-478b-b1ef-3d9035f7c7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:55.579165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:55.583054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:55.589993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 381/768] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:30:55.591132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7fd52-327e-478b-b1ef-3d9035f7c7de None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:55.598303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680eccb-427d-4b41-bd7c-687e5e804e5a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:55.642858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947f93b-1303-40fd-b509-ccb4f811b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:55.644074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947f93b-1303-40fd-b509-ccb4f811b6ce None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.644502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947f93b-1303-40fd-b509-ccb4f811b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.644987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947f93b-1303-40fd-b509-ccb4f811b6ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.653345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947f93b-1303-40fd-b509-ccb4f811b6ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:55.654860 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947f93b-1303-40fd-b509-ccb4f811b6ce None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a947f93b-1303-40fd-b509-ccb4f811b6ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:55.672124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:55.673092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:55.674009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:55.674009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:55.674009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:55.674009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:55.674009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:55.674009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:55.674009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:55.674009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:55.676526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 388/769] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:55.677653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947f93b-1303-40fd-b509-ccb4f811b6ce None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:55.730538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7fd52-327e-478b-b1ef-3d9035f7c7de None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:55.773215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67c59e-07f8-4d02-bf2b-ce081c8925d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:55.773895 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67c59e-07f8-4d02-bf2b-ce081c8925d5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.774181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67c59e-07f8-4d02-bf2b-ce081c8925d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.774460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67c59e-07f8-4d02-bf2b-ce081c8925d5 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.775177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67c59e-07f8-4d02-bf2b-ce081c8925d5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:55.775487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67c59e-07f8-4d02-bf2b-ce081c8925d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:55.776870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67c59e-07f8-4d02-bf2b-ce081c8925d5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:55.782271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 382/770] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:30:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:55.783113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67c59e-07f8-4d02-bf2b-ce081c8925d5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:55.812680 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947f93b-1303-40fd-b509-ccb4f811b6ce None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:55.845601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:55.850125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb311cd-802e-4089-a3f7-05b263eebefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:55.850908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb311cd-802e-4089-a3f7-05b263eebefb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.851477 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb311cd-802e-4089-a3f7-05b263eebefb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.851850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb311cd-802e-4089-a3f7-05b263eebefb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.852595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb311cd-802e-4089-a3f7-05b263eebefb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:55.854165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb311cd-802e-4089-a3f7-05b263eebefb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4bb311cd-802e-4089-a3f7-05b263eebefb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:55.867934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:55.869184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:30:55.870322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:55.873077 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 389/771] 10.222.0.88 () {56 vars in 1129 bytes} [Mon Jul 18 16:30:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:55.873943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb311cd-802e-4089-a3f7-05b263eebefb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:55.892790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:55.896326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67c59e-07f8-4d02-bf2b-ce081c8925d5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:55.942468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ff2ee-c052-4ec2-baf1-7a988b1b8c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:55.943138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ff2ee-c052-4ec2-baf1-7a988b1b8c15 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:55.943418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ff2ee-c052-4ec2-baf1-7a988b1b8c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:55.943689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ff2ee-c052-4ec2-baf1-7a988b1b8c15 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:55.944554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ff2ee-c052-4ec2-baf1-7a988b1b8c15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:55.944860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ff2ee-c052-4ec2-baf1-7a988b1b8c15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:55.946529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ff2ee-c052-4ec2-baf1-7a988b1b8c15 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:55.970967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 383/772] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:55.970967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ff2ee-c052-4ec2-baf1-7a988b1b8c15 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:56.007601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb311cd-802e-4089-a3f7-05b263eebefb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:56.071301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0940a84-0a7d-425b-82bc-ad8940617146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:56.072072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0940a84-0a7d-425b-82bc-ad8940617146 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:56.072413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0940a84-0a7d-425b-82bc-ad8940617146 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:56.072571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0940a84-0a7d-425b-82bc-ad8940617146 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:56.073546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0940a84-0a7d-425b-82bc-ad8940617146 None admin] RBAC: Adding query filter params (type=network) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:56.075734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0940a84-0a7d-425b-82bc-ad8940617146 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:56.077192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0940a84-0a7d-425b-82bc-ad8940617146 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:56.083215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 390/773] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:30:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:56.083966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0940a84-0a7d-425b-82bc-ad8940617146 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:56.108252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ff2ee-c052-4ec2-baf1-7a988b1b8c15 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:56.136833 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:56.161807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f52757-b8a8-48f6-ba0b-cdfb76bb0119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:56.162834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f52757-b8a8-48f6-ba0b-cdfb76bb0119 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:56.163315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f52757-b8a8-48f6-ba0b-cdfb76bb0119 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:56.163785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f52757-b8a8-48f6-ba0b-cdfb76bb0119 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:56.164706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f52757-b8a8-48f6-ba0b-cdfb76bb0119 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:56.165250 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f52757-b8a8-48f6-ba0b-cdfb76bb0119 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:56.166964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f52757-b8a8-48f6-ba0b-cdfb76bb0119 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:56.173211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 384/774] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Jul 18 16:30:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:56.174297 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f52757-b8a8-48f6-ba0b-cdfb76bb0119 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:56.185706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:56.212115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0940a84-0a7d-425b-82bc-ad8940617146 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:56.249935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e328e0-c2cd-440f-87cc-78a1d4c72574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:56.250901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e328e0-c2cd-440f-87cc-78a1d4c72574 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:56.251243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e328e0-c2cd-440f-87cc-78a1d4c72574 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:56.251529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e328e0-c2cd-440f-87cc-78a1d4c72574 None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/69ee145599384c6da2cba85a42832dcd/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:56.253912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e328e0-c2cd-440f-87cc-78a1d4c72574 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=69ee145599384c6da2cba85a42832dcd, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:56.255337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e328e0-c2cd-440f-87cc-78a1d4c72574 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:56.289507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f52757-b8a8-48f6-ba0b-cdfb76bb0119 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:56.325651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76061ed5-0465-4762-9961-5921a104750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:56.326246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061ed5-0465-4762-9961-5921a104750c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:56.326529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061ed5-0465-4762-9961-5921a104750c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:56.326805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061ed5-0465-4762-9961-5921a104750c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:56.327592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76061ed5-0465-4762-9961-5921a104750c None admin] RBAC: Adding query filter params (interface=public, service_id=5acd466f20ec49f6adbf5a7e966c7242, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:56.327953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76061ed5-0465-4762-9961-5921a104750c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:56.329417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76061ed5-0465-4762-9961-5921a104750c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:56.331838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 391/775] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:30:56 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/69ee145599384c6da2cba85a42832dcd/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:30:56.332915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e328e0-c2cd-440f-87cc-78a1d4c72574 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:56.347650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 385/776] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Jul 18 16:30:56 2022] GET /identity/v3/endpoints?service_id=5acd466f20ec49f6adbf5a7e966c7242&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:56.352487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76061ed5-0465-4762-9961-5921a104750c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:56.465944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e328e0-c2cd-440f-87cc-78a1d4c72574 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:56.478474 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:56.513002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f59e7-e6b3-48d5-a490-48a5e1b6163d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:56.514260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f59e7-e6b3-48d5-a490-48a5e1b6163d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:56.514805 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f59e7-e6b3-48d5-a490-48a5e1b6163d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:56.515422 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f59e7-e6b3-48d5-a490-48a5e1b6163d None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:56.516393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f59e7-e6b3-48d5-a490-48a5e1b6163d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:56.516990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f59e7-e6b3-48d5-a490-48a5e1b6163d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:56.518748 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f59e7-e6b3-48d5-a490-48a5e1b6163d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:56.529343 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:56.529612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 392/777] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Jul 18 16:30:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:56.533184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f59e7-e6b3-48d5-a490-48a5e1b6163d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:56.622622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76061ed5-0465-4762-9961-5921a104750c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:56.665807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f59e7-e6b3-48d5-a490-48a5e1b6163d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:56.672670 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4880f9-60d4-4c40-a1ef-52c5371036db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:56.673481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4880f9-60d4-4c40-a1ef-52c5371036db None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:56.673838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4880f9-60d4-4c40-a1ef-52c5371036db None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:56.674189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4880f9-60d4-4c40-a1ef-52c5371036db None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:56.675073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4880f9-60d4-4c40-a1ef-52c5371036db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:56.676551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4880f9-60d4-4c40-a1ef-52c5371036db None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:56.683358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 386/778] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Jul 18 16:30:56 2022] GET /identity/v3/services => generated 772 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:56.713824 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb44d8-de4f-46bd-ae28-d374164873fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:56.714553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb44d8-de4f-46bd-ae28-d374164873fd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:56.714838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb44d8-de4f-46bd-ae28-d374164873fd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:56.715112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb44d8-de4f-46bd-ae28-d374164873fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:56.716457 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb44d8-de4f-46bd-ae28-d374164873fd None admin] RBAC: Adding query filter params (interface=public, service_id=a678183124504dd1b4813ec0df1cc3d4, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:56.716766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb44d8-de4f-46bd-ae28-d374164873fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:56.718231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb44d8-de4f-46bd-ae28-d374164873fd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:56.727688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 393/779] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Jul 18 16:30:56 2022] GET /identity/v3/endpoints?service_id=a678183124504dd1b4813ec0df1cc3d4&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:56.733850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4880f9-60d4-4c40-a1ef-52c5371036db None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:56.871862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4880f9-60d4-4c40-a1ef-52c5371036db None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:56.888482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:56.925250 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d696c06-98a5-434f-bcdb-2e7c33b08c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:56.925942 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d696c06-98a5-434f-bcdb-2e7c33b08c3b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:56.926224 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d696c06-98a5-434f-bcdb-2e7c33b08c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:56.926500 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d696c06-98a5-434f-bcdb-2e7c33b08c3b None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:56.927201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d696c06-98a5-434f-bcdb-2e7c33b08c3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:56.928634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d696c06-98a5-434f-bcdb-2e7c33b08c3b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:56.935739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 387/780] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Jul 18 16:30:56 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:57.211524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eb23b-6a60-4538-95b0-59f878876c01 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:57.212508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eb23b-6a60-4538-95b0-59f878876c01 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:57.213044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8eb23b-6a60-4538-95b0-59f878876c01 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:57.214878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 394/781] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:57.220872 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511376e9-6901-4fbe-91b1-938cdfe58284 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:57.221359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511376e9-6901-4fbe-91b1-938cdfe58284 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:57.221546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511376e9-6901-4fbe-91b1-938cdfe58284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:57.242960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-511376e9-6901-4fbe-91b1-938cdfe58284 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:57.244163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-511376e9-6901-4fbe-91b1-938cdfe58284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:57.326190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 388/782] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:30:57.360109 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8eb23b-6a60-4538-95b0-59f878876c01 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:57.451601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:57.509587 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8eb23b-6a60-4538-95b0-59f878876c01 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:57.555560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2051d23-c348-4352-b682-3e1f381e22fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:57.556371 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2051d23-c348-4352-b682-3e1f381e22fa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:57.556652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2051d23-c348-4352-b682-3e1f381e22fa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:57.556945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2051d23-c348-4352-b682-3e1f381e22fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:57.574404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2051d23-c348-4352-b682-3e1f381e22fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-b2051d23-c348-4352-b682-3e1f381e22fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:57.582705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:57.583641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:57.585110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 395/783] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:57.589884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511376e9-6901-4fbe-91b1-938cdfe58284 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:57.767780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511376e9-6901-4fbe-91b1-938cdfe58284 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:57.770246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:57.814522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03059152-bc5b-4e60-98d6-2d3c6402bccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:57.815215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03059152-bc5b-4e60-98d6-2d3c6402bccc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:57.815500 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03059152-bc5b-4e60-98d6-2d3c6402bccc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:57.815773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03059152-bc5b-4e60-98d6-2d3c6402bccc None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:57.816535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03059152-bc5b-4e60-98d6-2d3c6402bccc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:57.816838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03059152-bc5b-4e60-98d6-2d3c6402bccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:57.818290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03059152-bc5b-4e60-98d6-2d3c6402bccc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:57.832870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 389/784] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:57.838194 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2051d23-c348-4352-b682-3e1f381e22fa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:57.857004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c76c8-0c47-4fb4-8148-3f18859412b8 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:57.857342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c76c8-0c47-4fb4-8148-3f18859412b8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:57.857578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c76c8-0c47-4fb4-8148-3f18859412b8 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:57.858961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 390/785] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:57.866814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fa19b-9ca4-43d5-8286-da1645754a6d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:57.867350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fa19b-9ca4-43d5-8286-da1645754a6d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:57.867644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fa19b-9ca4-43d5-8286-da1645754a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:57.888569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d79fa19b-9ca4-43d5-8286-da1645754a6d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:57.889810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d79fa19b-9ca4-43d5-8286-da1645754a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:58.009665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 391/786] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:30:58.027445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2051d23-c348-4352-b682-3e1f381e22fa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:58.031385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fa19b-9ca4-43d5-8286-da1645754a6d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:58.069384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:58.093621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedb213-a4bc-4157-b53d-26db5a5d1bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:58.094442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedb213-a4bc-4157-b53d-26db5a5d1bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:58.094727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedb213-a4bc-4157-b53d-26db5a5d1bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:58.095004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eedb213-a4bc-4157-b53d-26db5a5d1bf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:58.108327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedb213-a4bc-4157-b53d-26db5a5d1bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:58.110128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eedb213-a4bc-4157-b53d-26db5a5d1bf7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2eedb213-a4bc-4157-b53d-26db5a5d1bf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:58.130045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:58.130989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:58.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:58.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:58.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:58.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:58.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 396/787] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:58.138561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedb213-a4bc-4157-b53d-26db5a5d1bf7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:58.171526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:58.183252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fa19b-9ca4-43d5-8286-da1645754a6d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:58.254223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10538861-cb9a-4cf2-bd75-343aabc058a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:58.255038 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10538861-cb9a-4cf2-bd75-343aabc058a5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:58.255322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10538861-cb9a-4cf2-bd75-343aabc058a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:58.255596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10538861-cb9a-4cf2-bd75-343aabc058a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:58.270125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eedb213-a4bc-4157-b53d-26db5a5d1bf7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:58.280371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10538861-cb9a-4cf2-bd75-343aabc058a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-10538861-cb9a-4cf2-bd75-343aabc058a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:58.288108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:58.289087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:58.289930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 392/788] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:58.297863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10538861-cb9a-4cf2-bd75-343aabc058a5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:58.312535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bed8a9-ee38-4da9-8f26-8013e4fe2877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:58.313574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bed8a9-ee38-4da9-8f26-8013e4fe2877 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:58.314073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bed8a9-ee38-4da9-8f26-8013e4fe2877 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:58.314610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bed8a9-ee38-4da9-8f26-8013e4fe2877 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:58.315540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bed8a9-ee38-4da9-8f26-8013e4fe2877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:58.316086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bed8a9-ee38-4da9-8f26-8013e4fe2877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:58.317691 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bed8a9-ee38-4da9-8f26-8013e4fe2877 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:58.331216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 397/789] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:58.336297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bed8a9-ee38-4da9-8f26-8013e4fe2877 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:58.389464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:58.451834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bed8a9-ee38-4da9-8f26-8013e4fe2877 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:58.487519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dcc35-470f-4175-84cf-d3d5cf5167c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:58.488665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dcc35-470f-4175-84cf-d3d5cf5167c8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:58.489218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dcc35-470f-4175-84cf-d3d5cf5167c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:58.489835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dcc35-470f-4175-84cf-d3d5cf5167c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:58.507075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dcc35-470f-4175-84cf-d3d5cf5167c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:58.509881 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dcc35-470f-4175-84cf-d3d5cf5167c8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-059dcc35-470f-4175-84cf-d3d5cf5167c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:58.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:58.533747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:58.534692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:58.534692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:58.534692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:58.534692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:58.534692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:58.534692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:58.534692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:58.534692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:58.534692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:30:58.534692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:58.537330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 398/790] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:58.543140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dcc35-470f-4175-84cf-d3d5cf5167c8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:58.563106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10538861-cb9a-4cf2-bd75-343aabc058a5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:58.644302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ef848-7b41-4239-9bdb-11f8d6926e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:58.645399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ef848-7b41-4239-9bdb-11f8d6926e75 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:58.645926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ef848-7b41-4239-9bdb-11f8d6926e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:58.646400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ef848-7b41-4239-9bdb-11f8d6926e75 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:58.647356 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ef848-7b41-4239-9bdb-11f8d6926e75 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:58.651295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ef848-7b41-4239-9bdb-11f8d6926e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:58.652991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ef848-7b41-4239-9bdb-11f8d6926e75 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:58.666431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 393/791] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:30:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:58.673105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:58.673902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ef848-7b41-4239-9bdb-11f8d6926e75 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:58.688383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dcc35-470f-4175-84cf-d3d5cf5167c8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:58.728341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:58.759455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846ef67-e787-4f87-98f4-a439f8ad2e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:58.760138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846ef67-e787-4f87-98f4-a439f8ad2e2f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:58.760467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846ef67-e787-4f87-98f4-a439f8ad2e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:58.760749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846ef67-e787-4f87-98f4-a439f8ad2e2f None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:58.761626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846ef67-e787-4f87-98f4-a439f8ad2e2f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:58.761936 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846ef67-e787-4f87-98f4-a439f8ad2e2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:58.763711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846ef67-e787-4f87-98f4-a439f8ad2e2f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:58.802157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 399/792] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Jul 18 16:30:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:30:58.805739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846ef67-e787-4f87-98f4-a439f8ad2e2f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:58.840845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ef848-7b41-4239-9bdb-11f8d6926e75 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:58.879472 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e98c9-1cc9-40d4-a58c-b81f9c8e6fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:58.880300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e98c9-1cc9-40d4-a58c-b81f9c8e6fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:58.880582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e98c9-1cc9-40d4-a58c-b81f9c8e6fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:58.880853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e98c9-1cc9-40d4-a58c-b81f9c8e6fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:58.895833 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e98c9-1cc9-40d4-a58c-b81f9c8e6fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:58.897898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e98c9-1cc9-40d4-a58c-b81f9c8e6fe2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-984e98c9-1cc9-40d4-a58c-b81f9c8e6fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:58.921184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:58.922116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:58.923019 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:58.923019 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:58.923019 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:58.923019 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:58.926053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 394/793] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:58.927648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9116af-9fd8-4c9d-822d-9b83b7a67488 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:58.928075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9116af-9fd8-4c9d-822d-9b83b7a67488 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:58.928496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9116af-9fd8-4c9d-822d-9b83b7a67488 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:58.934879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 395/794] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:58.935743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9116af-9fd8-4c9d-822d-9b83b7a67488 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:58.949199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846ef67-e787-4f87-98f4-a439f8ad2e2f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:58.987470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1662e2c-5e51-4da2-a0f2-b5309ec28a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:58.988629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1662e2c-5e51-4da2-a0f2-b5309ec28a27 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:58.989339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1662e2c-5e51-4da2-a0f2-b5309ec28a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:58.989920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1662e2c-5e51-4da2-a0f2-b5309ec28a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.002186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1662e2c-5e51-4da2-a0f2-b5309ec28a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:59.004238 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1662e2c-5e51-4da2-a0f2-b5309ec28a27 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-c1662e2c-5e51-4da2-a0f2-b5309ec28a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:59.024103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:59.025087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:59.025988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:59.025988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:59.025988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:59.025988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:59.028561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 400/795] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:59.029080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:59.030571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f61a6-58cb-4374-a66d-47b3a97e361a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.031063 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f61a6-58cb-4374-a66d-47b3a97e361a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.031531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f61a6-58cb-4374-a66d-47b3a97e361a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.043691 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:59.050911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-841f61a6-58cb-4374-a66d-47b3a97e361a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:59.052525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-841f61a6-58cb-4374-a66d-47b3a97e361a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:59.070028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9116af-9fd8-4c9d-822d-9b83b7a67488 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:59.114316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2175ac-964f-441b-842b-70da1e955469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:59.114980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2175ac-964f-441b-842b-70da1e955469 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.115268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2175ac-964f-441b-842b-70da1e955469 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.115548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2175ac-964f-441b-842b-70da1e955469 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.116283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2175ac-964f-441b-842b-70da1e955469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:59.116591 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2175ac-964f-441b-842b-70da1e955469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:59.118783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2175ac-964f-441b-842b-70da1e955469 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:59.134075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 401/796] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:30:59.135297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f61a6-58cb-4374-a66d-47b3a97e361a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:59.142274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 396/797] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:59.159034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2175ac-964f-441b-842b-70da1e955469 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:59.275031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2175ac-964f-441b-842b-70da1e955469 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:59.281470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f61a6-58cb-4374-a66d-47b3a97e361a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:59.311068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:59.317305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b3b8a-8c83-46e3-add8-214c2d491a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:59.318128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b3b8a-8c83-46e3-add8-214c2d491a25 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.318411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b3b8a-8c83-46e3-add8-214c2d491a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.318839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b3b8a-8c83-46e3-add8-214c2d491a25 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.324670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab90ec-5244-4ac7-b56e-8badf6af8668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:59.325574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab90ec-5244-4ac7-b56e-8badf6af8668 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.326050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab90ec-5244-4ac7-b56e-8badf6af8668 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.326333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab90ec-5244-4ac7-b56e-8badf6af8668 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.326551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b3b8a-8c83-46e3-add8-214c2d491a25 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:59.327087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab90ec-5244-4ac7-b56e-8badf6af8668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:59.327395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab90ec-5244-4ac7-b56e-8badf6af8668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:59.328012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b3b8a-8c83-46e3-add8-214c2d491a25 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:59.328755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab90ec-5244-4ac7-b56e-8badf6af8668 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:59.331289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:59.341027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 402/798] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:59.342267 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab90ec-5244-4ac7-b56e-8badf6af8668 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-829b3b8a-8c83-46e3-add8-214c2d491a25 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:30:59.343925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:59.344999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:59.346732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:59.346732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:59.346732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:30:59.346732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:30:59.346732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:30:59.346732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:30:59.346732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:30:59.346732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:30:59.346732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:30:59.346732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:59.346732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 397/799] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Jul 18 16:30:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:30:59.347361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e8803-c6e3-4ea6-b3d3-02fa6ed6f23e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.347361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e8803-c6e3-4ea6-b3d3-02fa6ed6f23e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.347361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e8803-c6e3-4ea6-b3d3-02fa6ed6f23e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.348808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 398/800] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:59.351662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e8803-c6e3-4ea6-b3d3-02fa6ed6f23e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:59.455686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab90ec-5244-4ac7-b56e-8badf6af8668 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:59.461701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e8803-c6e3-4ea6-b3d3-02fa6ed6f23e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:59.491825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ec490-6896-4e25-be70-52d259bf3880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:59.492894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ec490-6896-4e25-be70-52d259bf3880 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.493206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ec490-6896-4e25-be70-52d259bf3880 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.493494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ec490-6896-4e25-be70-52d259bf3880 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.499140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317b5fb-239d-440b-9b73-97d9b69ea64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:59.500232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317b5fb-239d-440b-9b73-97d9b69ea64b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.500704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317b5fb-239d-440b-9b73-97d9b69ea64b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.501186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317b5fb-239d-440b-9b73-97d9b69ea64b None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.502230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317b5fb-239d-440b-9b73-97d9b69ea64b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:59.502733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317b5fb-239d-440b-9b73-97d9b69ea64b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:59.504482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317b5fb-239d-440b-9b73-97d9b69ea64b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:59.504600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ec490-6896-4e25-be70-52d259bf3880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-e56ec490-6896-4e25-be70-52d259bf3880 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:30:59.512319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:59.513366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:30:59.514202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:59.518487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 403/801] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:30:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:59.519610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ec490-6896-4e25-be70-52d259bf3880 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:59.526586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 399/802] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Jul 18 16:30:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:59.528222 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c41f4-fb44-4809-8ff6-97fab00eb331 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.528699 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c41f4-fb44-4809-8ff6-97fab00eb331 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.529186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c41f4-fb44-4809-8ff6-97fab00eb331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.547901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f80c41f4-fb44-4809-8ff6-97fab00eb331 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:30:59.549151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f80c41f4-fb44-4809-8ff6-97fab00eb331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:30:59.592341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:59.601480 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:59.626208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 400/803] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:30:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:30:59.627056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c41f4-fb44-4809-8ff6-97fab00eb331 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:59.648976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ec490-6896-4e25-be70-52d259bf3880 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:59.683532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06551d-c303-4582-8b74-f36e524f87e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:59.684356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06551d-c303-4582-8b74-f36e524f87e8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.684631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06551d-c303-4582-8b74-f36e524f87e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.684952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06551d-c303-4582-8b74-f36e524f87e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.691815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06551d-c303-4582-8b74-f36e524f87e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:59.693335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06551d-c303-4582-8b74-f36e524f87e8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4c06551d-c303-4582-8b74-f36e524f87e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:30:59.708969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:59.709983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:59.710868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:30:59.710868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:30:59.710868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:30:59.710868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:30:59.710868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:30:59.710868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:30:59.710868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:30:59.710868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:59.710868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 404/804] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:59.711379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06551d-c303-4582-8b74-f36e524f87e8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:59.737692 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c41f4-fb44-4809-8ff6-97fab00eb331 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:59.773120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cfedb-3a5c-4491-84da-1b29d4842c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:59.773893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cfedb-3a5c-4491-84da-1b29d4842c44 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.774254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cfedb-3a5c-4491-84da-1b29d4842c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.774601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cfedb-3a5c-4491-84da-1b29d4842c44 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.775367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cfedb-3a5c-4491-84da-1b29d4842c44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:30:59.775680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cfedb-3a5c-4491-84da-1b29d4842c44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:59.777172 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34cfedb-3a5c-4491-84da-1b29d4842c44 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:59.791788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 401/805] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:30:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:30:59.791788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88c89c-bdcd-4452-b7fc-b15bf38b2e2c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.791788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88c89c-bdcd-4452-b7fc-b15bf38b2e2c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.791788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88c89c-bdcd-4452-b7fc-b15bf38b2e2c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.791788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 402/806] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:30:59.794451 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88c89c-bdcd-4452-b7fc-b15bf38b2e2c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:59.818555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06551d-c303-4582-8b74-f36e524f87e8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:59.837264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:59.840755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:30:59.857555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306c5fe-f12b-449f-8e21-1162499026eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:59.858314 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306c5fe-f12b-449f-8e21-1162499026eb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.858561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306c5fe-f12b-449f-8e21-1162499026eb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.858836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306c5fe-f12b-449f-8e21-1162499026eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.869380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9306c5fe-f12b-449f-8e21-1162499026eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:59.870865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9306c5fe-f12b-449f-8e21-1162499026eb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9306c5fe-f12b-449f-8e21-1162499026eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:30:59.888720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:59.889799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:59.890888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:30:59.890888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:30:59.890888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:30:59.890888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:30:59.891261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 405/807] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:59.892909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306c5fe-f12b-449f-8e21-1162499026eb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:30:59.901243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88c89c-bdcd-4452-b7fc-b15bf38b2e2c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:30:59.940184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ab55b-ce30-40f2-aa79-2344708cd26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:30:59.941021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ab55b-ce30-40f2-aa79-2344708cd26d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:30:59.941308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ab55b-ce30-40f2-aa79-2344708cd26d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:30:59.941587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ab55b-ce30-40f2-aa79-2344708cd26d None admin] PATH_INFO: `/v3/services/network` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:30:59.942251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ab55b-ce30-40f2-aa79-2344708cd26d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:30:59.943759 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ab55b-ce30-40f2-aa79-2344708cd26d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-319ab55b-ce30-40f2-aa79-2344708cd26d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:30:59.956073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:30:59.957344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:30:59.958302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 403/808] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:30:59 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:30:59.959265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ab55b-ce30-40f2-aa79-2344708cd26d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:00.002228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306c5fe-f12b-449f-8e21-1162499026eb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.037291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a6153-552a-4437-9280-74b41b1bcdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.037966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a6153-552a-4437-9280-74b41b1bcdcd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.038256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a6153-552a-4437-9280-74b41b1bcdcd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.038538 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a6153-552a-4437-9280-74b41b1bcdcd None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.039333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a6153-552a-4437-9280-74b41b1bcdcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:00.039642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a6153-552a-4437-9280-74b41b1bcdcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.041106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a6153-552a-4437-9280-74b41b1bcdcd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:00.053539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 406/809] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:30:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:00.054876 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3d979-fd64-4894-b77e-80bcea54e606 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.055176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3d979-fd64-4894-b77e-80bcea54e606 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.055458 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3d979-fd64-4894-b77e-80bcea54e606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.061459 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ab55b-ce30-40f2-aa79-2344708cd26d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.064747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:00.075008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1ef3d979-fd64-4894-b77e-80bcea54e606 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:00.085271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1ef3d979-fd64-4894-b77e-80bcea54e606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:00.098123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700b911-758a-4630-9868-ddcf2b45e92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.098920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700b911-758a-4630-9868-ddcf2b45e92b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.099201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700b911-758a-4630-9868-ddcf2b45e92b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.099478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700b911-758a-4630-9868-ddcf2b45e92b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.099875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:00.110054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700b911-758a-4630-9868-ddcf2b45e92b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.111437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700b911-758a-4630-9868-ddcf2b45e92b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-9700b911-758a-4630-9868-ddcf2b45e92b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:00.131884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:00.143565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:00.144645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:00.144645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:00.144645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:00.144645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:00.144645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 404/810] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:00.144645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700b911-758a-4630-9868-ddcf2b45e92b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:00.146561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 407/811] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:00.147514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3d979-fd64-4894-b77e-80bcea54e606 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:00.250532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3d979-fd64-4894-b77e-80bcea54e606 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.254308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700b911-758a-4630-9868-ddcf2b45e92b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.285556 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4ad39-c1cd-450a-b1a5-0fc4800be865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.286375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4ad39-c1cd-450a-b1a5-0fc4800be865 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.286663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4ad39-c1cd-450a-b1a5-0fc4800be865 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.286940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4ad39-c1cd-450a-b1a5-0fc4800be865 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.287655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4ad39-c1cd-450a-b1a5-0fc4800be865 None admin] RBAC: Adding query filter params (name=network) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:00.287965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4ad39-c1cd-450a-b1a5-0fc4800be865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.289011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055428e3-d3fc-4064-bdb2-caee573a53dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.289393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4ad39-c1cd-450a-b1a5-0fc4800be865 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:00.289687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055428e3-d3fc-4064-bdb2-caee573a53dc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.289973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055428e3-d3fc-4064-bdb2-caee573a53dc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.290245 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055428e3-d3fc-4064-bdb2-caee573a53dc None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.290974 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055428e3-d3fc-4064-bdb2-caee573a53dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:00.291280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055428e3-d3fc-4064-bdb2-caee573a53dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.292630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055428e3-d3fc-4064-bdb2-caee573a53dc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:00.294900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 408/812] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:31:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:00.295490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4ad39-c1cd-450a-b1a5-0fc4800be865 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:00.305072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 405/813] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:00.305898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055428e3-d3fc-4064-bdb2-caee573a53dc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:00.325293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:00.325476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:00.409685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055428e3-d3fc-4064-bdb2-caee573a53dc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.410873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4ad39-c1cd-450a-b1a5-0fc4800be865 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.444787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bf29e-17c9-4104-905e-23d6137266a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.445485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bf29e-17c9-4104-905e-23d6137266a7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.445760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed5916-7953-438d-8396-0acf5ceaf5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.445900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bf29e-17c9-4104-905e-23d6137266a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.446193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bf29e-17c9-4104-905e-23d6137266a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.446647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed5916-7953-438d-8396-0acf5ceaf5fb None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.446817 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bf29e-17c9-4104-905e-23d6137266a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:00.446922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed5916-7953-438d-8396-0acf5ceaf5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.447112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bf29e-17c9-4104-905e-23d6137266a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.447214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed5916-7953-438d-8396-0acf5ceaf5fb None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/bf341f1bb8944a3e9331678100b941a1/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.448488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bf29e-17c9-4104-905e-23d6137266a7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:00.449165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed5916-7953-438d-8396-0acf5ceaf5fb None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=bf341f1bb8944a3e9331678100b941a1, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.450632 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed5916-7953-438d-8396-0acf5ceaf5fb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:00.459954 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 406/814] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:00.460793 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bf29e-17c9-4104-905e-23d6137266a7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:00.534264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 409/815] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:31:00 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/bf341f1bb8944a3e9331678100b941a1/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:31:00.535111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed5916-7953-438d-8396-0acf5ceaf5fb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:00.545771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:00.573119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bf29e-17c9-4104-905e-23d6137266a7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.613504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c5be0-9ded-4324-8f6c-38a1196d2a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.614338 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c5be0-9ded-4324-8f6c-38a1196d2a09 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.614623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c5be0-9ded-4324-8f6c-38a1196d2a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.614938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c5be0-9ded-4324-8f6c-38a1196d2a09 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.615577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c5be0-9ded-4324-8f6c-38a1196d2a09 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.617013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c5be0-9ded-4324-8f6c-38a1196d2a09 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f02c5be0-9ded-4324-8f6c-38a1196d2a09 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:00.630483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:00.631691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 16:31:00.632746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:00.633770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:00.636931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 407/816] 10.222.0.88 () {56 vars in 1129 bytes} [Mon Jul 18 16:31:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:00.638016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c5be0-9ded-4324-8f6c-38a1196d2a09 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:00.660723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed5916-7953-438d-8396-0acf5ceaf5fb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.760111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c5be0-9ded-4324-8f6c-38a1196d2a09 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.799231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26eaf0-ed0a-4061-b034-38c1199b904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.800035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26eaf0-ed0a-4061-b034-38c1199b904d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.800321 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26eaf0-ed0a-4061-b034-38c1199b904d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.852152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26eaf0-ed0a-4061-b034-38c1199b904d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.852152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26eaf0-ed0a-4061-b034-38c1199b904d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.852152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26eaf0-ed0a-4061-b034-38c1199b904d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:00.852152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:00.852535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133fa18-a340-4723-993a-b15c905b5ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.852535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133fa18-a340-4723-993a-b15c905b5ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.852535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133fa18-a340-4723-993a-b15c905b5ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.852535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133fa18-a340-4723-993a-b15c905b5ce7 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.852866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133fa18-a340-4723-993a-b15c905b5ce7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:00.852866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133fa18-a340-4723-993a-b15c905b5ce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.852866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133fa18-a340-4723-993a-b15c905b5ce7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ed26eaf0-ed0a-4061-b034-38c1199b904d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:31:00.853139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:00.854034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:00.855035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:31:00.855035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:31:00.855035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:31:00.855035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:31:00.855035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:31:00.855035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:31:00.855035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:00.855035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:00.855540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 410/817] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:31:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:00.855615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 408/818] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:00.855739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133fa18-a340-4723-993a-b15c905b5ce7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:00.856326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26eaf0-ed0a-4061-b034-38c1199b904d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:00.945009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26eaf0-ed0a-4061-b034-38c1199b904d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.946287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133fa18-a340-4723-993a-b15c905b5ce7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:00.982415 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c341dc-124b-4c3f-b645-7d9258c4aecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.982851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b81b9e-8fa5-40ac-adbd-8c3d1f774828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:00.983089 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c341dc-124b-4c3f-b645-7d9258c4aecd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.983372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c341dc-124b-4c3f-b645-7d9258c4aecd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.983648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b81b9e-8fa5-40ac-adbd-8c3d1f774828 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:00.983745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c341dc-124b-4c3f-b645-7d9258c4aecd None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.983933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b81b9e-8fa5-40ac-adbd-8c3d1f774828 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:00.984210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b81b9e-8fa5-40ac-adbd-8c3d1f774828 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:00.984369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c341dc-124b-4c3f-b645-7d9258c4aecd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:00.984654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c341dc-124b-4c3f-b645-7d9258c4aecd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.986072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c341dc-124b-4c3f-b645-7d9258c4aecd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:00.992384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 409/819] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Jul 18 16:31:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:00.993203 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c341dc-124b-4c3f-b645-7d9258c4aecd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:00.998435 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b81b9e-8fa5-40ac-adbd-8c3d1f774828 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:00.999908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b81b9e-8fa5-40ac-adbd-8c3d1f774828 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-52b81b9e-8fa5-40ac-adbd-8c3d1f774828 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:31:01.015559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:01.016727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 411/820] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Jul 18 16:31:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:01.017808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b81b9e-8fa5-40ac-adbd-8c3d1f774828 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:01.029501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:01.065807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:01.105140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c341dc-124b-4c3f-b645-7d9258c4aecd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:01.130335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b81b9e-8fa5-40ac-adbd-8c3d1f774828 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:01.143263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03233750-aac3-4a9e-9e15-d452a60ea322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:01.144041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233750-aac3-4a9e-9e15-d452a60ea322 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:01.144408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233750-aac3-4a9e-9e15-d452a60ea322 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:01.144742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03233750-aac3-4a9e-9e15-d452a60ea322 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:01.145592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233750-aac3-4a9e-9e15-d452a60ea322 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:01.147049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03233750-aac3-4a9e-9e15-d452a60ea322 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:01.169760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bed680-d458-4ae2-b6dd-2a6d69f0008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:01.170425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bed680-d458-4ae2-b6dd-2a6d69f0008f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:01.170713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bed680-d458-4ae2-b6dd-2a6d69f0008f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:01.171557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bed680-d458-4ae2-b6dd-2a6d69f0008f None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:01.171830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bed680-d458-4ae2-b6dd-2a6d69f0008f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:01.172148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bed680-d458-4ae2-b6dd-2a6d69f0008f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:01.173614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bed680-d458-4ae2-b6dd-2a6d69f0008f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:01.186531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 412/821] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:01.187307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bed680-d458-4ae2-b6dd-2a6d69f0008f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:01.254741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 410/822] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Jul 18 16:31:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:01.255963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03233750-aac3-4a9e-9e15-d452a60ea322 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:01.266816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:01.309532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bed680-d458-4ae2-b6dd-2a6d69f0008f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:01.357745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7198c-d2a8-46d4-9a8c-7da31511d0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:01.358587 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7198c-d2a8-46d4-9a8c-7da31511d0ab None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:01.358978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7198c-d2a8-46d4-9a8c-7da31511d0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:01.359475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7198c-d2a8-46d4-9a8c-7da31511d0ab None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:01.360458 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7198c-d2a8-46d4-9a8c-7da31511d0ab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:01.360850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7198c-d2a8-46d4-9a8c-7da31511d0ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:01.362524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7198c-d2a8-46d4-9a8c-7da31511d0ab None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:01.373999 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 413/823] 10.222.0.88 () {56 vars in 1124 bytes} [Mon Jul 18 16:31:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:01.375060 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7198c-d2a8-46d4-9a8c-7da31511d0ab None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:01.402672 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03233750-aac3-4a9e-9e15-d452a60ea322 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:01.434440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:01.456054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b89fd-828b-470c-bb5f-43bc5d0d2fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:01.457074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b89fd-828b-470c-bb5f-43bc5d0d2fff None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:01.457572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b89fd-828b-470c-bb5f-43bc5d0d2fff None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:01.457856 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b89fd-828b-470c-bb5f-43bc5d0d2fff None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:01.458704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b89fd-828b-470c-bb5f-43bc5d0d2fff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:01.459015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b89fd-828b-470c-bb5f-43bc5d0d2fff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:01.460459 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b89fd-828b-470c-bb5f-43bc5d0d2fff None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:01.484228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 411/824] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Jul 18 16:31:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:01.485554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b89fd-828b-470c-bb5f-43bc5d0d2fff None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:01.503400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7198c-d2a8-46d4-9a8c-7da31511d0ab None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:01.536531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:01.548232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1c757-60a6-4099-893d-7e67eadff07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:01.549186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1c757-60a6-4099-893d-7e67eadff07a None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:01.549470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1c757-60a6-4099-893d-7e67eadff07a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:01.549748 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1c757-60a6-4099-893d-7e67eadff07a None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/3f38abef63e24567a995b30c144177fa/roles/3fc59073330c4557ad178629386b0852` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:01.569656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1c757-60a6-4099-893d-7e67eadff07a None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=3f38abef63e24567a995b30c144177fa, role_id=3fc59073330c4557ad178629386b0852)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:01.574126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1c757-60a6-4099-893d-7e67eadff07a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:01.602761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b89fd-828b-470c-bb5f-43bc5d0d2fff None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:01.638968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7eda9-3f29-4140-aaba-3fc7c53e9782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:01.639721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7eda9-3f29-4140-aaba-3fc7c53e9782 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:01.640014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7eda9-3f29-4140-aaba-3fc7c53e9782 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:01.640293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7eda9-3f29-4140-aaba-3fc7c53e9782 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:01.641126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7eda9-3f29-4140-aaba-3fc7c53e9782 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:01.642522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7eda9-3f29-4140-aaba-3fc7c53e9782 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:01.653320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 414/825] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:31:01 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/3f38abef63e24567a995b30c144177fa/roles/3fc59073330c4557ad178629386b0852 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:31:01.654379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1c757-60a6-4099-893d-7e67eadff07a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:01.741118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 412/826] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Jul 18 16:31:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:01.812616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1c757-60a6-4099-893d-7e67eadff07a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:01.864175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f13b1-b299-4d04-bc3b-e895263a112a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:01.864938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f13b1-b299-4d04-bc3b-e895263a112a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:01.867833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f13b1-b299-4d04-bc3b-e895263a112a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:01.867833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f13b1-b299-4d04-bc3b-e895263a112a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:01.913253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:01.953776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f13b1-b299-4d04-bc3b-e895263a112a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:01.955252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f13b1-b299-4d04-bc3b-e895263a112a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:02.065363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 415/827] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:31:02.070991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7eda9-3f29-4140-aaba-3fc7c53e9782 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:02.114550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:02.187625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7eda9-3f29-4140-aaba-3fc7c53e9782 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:02.228036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f62482-338e-4220-b3d6-f667259eca06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:02.229258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f62482-338e-4220-b3d6-f667259eca06 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:02.229787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f62482-338e-4220-b3d6-f667259eca06 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:02.230262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f62482-338e-4220-b3d6-f667259eca06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:02.241489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f62482-338e-4220-b3d6-f667259eca06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:02.243542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f62482-338e-4220-b3d6-f667259eca06 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d7f62482-338e-4220-b3d6-f667259eca06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:02.264557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:02.265555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:02.266441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:02.266441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:02.266441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:02.266441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:02.268753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 413/828] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:02.280917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f13b1-b299-4d04-bc3b-e895263a112a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:02.352514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:02.384028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f13b1-b299-4d04-bc3b-e895263a112a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:02.422091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bc086-3de7-4e21-9bd5-635b97b78e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:02.422766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bc086-3de7-4e21-9bd5-635b97b78e73 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:02.423050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bc086-3de7-4e21-9bd5-635b97b78e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:02.423332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bc086-3de7-4e21-9bd5-635b97b78e73 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:02.424068 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bc086-3de7-4e21-9bd5-635b97b78e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:02.424372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bc086-3de7-4e21-9bd5-635b97b78e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:02.425788 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bc086-3de7-4e21-9bd5-635b97b78e73 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:02.438613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 416/829] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:02.443696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f62482-338e-4220-b3d6-f667259eca06 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:02.539724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:02.555472 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f62482-338e-4220-b3d6-f667259eca06 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:02.592596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb9caf-1808-42a2-994d-d1728e5fe663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:02.594103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb9caf-1808-42a2-994d-d1728e5fe663 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:02.594613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb9caf-1808-42a2-994d-d1728e5fe663 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:02.595085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb9caf-1808-42a2-994d-d1728e5fe663 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:02.602715 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb9caf-1808-42a2-994d-d1728e5fe663 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:02.604419 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb9caf-1808-42a2-994d-d1728e5fe663 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:02.614795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776e6a6-5560-4501-b49c-fd8149eb79c2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:02.615084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776e6a6-5560-4501-b49c-fd8149eb79c2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:02.615373 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776e6a6-5560-4501-b49c-fd8149eb79c2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:02.616896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 417/830] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-9efb9caf-1808-42a2-994d-d1728e5fe663 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:31:02.619153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:02.619996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:02.621003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:31:02.621003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:31:02.621003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:31:02.621003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:31:02.621003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:31:02.621003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:31:02.621003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:02.621003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:02.621003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 414/831] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:02.627596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce6009-de83-4dd1-951a-fd1f18a72901 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:02.627596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce6009-de83-4dd1-951a-fd1f18a72901 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:02.627596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce6009-de83-4dd1-951a-fd1f18a72901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:02.629786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb9caf-1808-42a2-994d-d1728e5fe663 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:02.644174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-80ce6009-de83-4dd1-951a-fd1f18a72901 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:02.645540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-80ce6009-de83-4dd1-951a-fd1f18a72901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:02.725165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 418/832] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:02.741577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce6009-de83-4dd1-951a-fd1f18a72901 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:02.763223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb9caf-1808-42a2-994d-d1728e5fe663 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:02.808341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:02.813373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba865dbc-d4d1-4f7f-826c-6b44cddc59f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:02.814821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba865dbc-d4d1-4f7f-826c-6b44cddc59f5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:02.815568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba865dbc-d4d1-4f7f-826c-6b44cddc59f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:02.816054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba865dbc-d4d1-4f7f-826c-6b44cddc59f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:02.817229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba865dbc-d4d1-4f7f-826c-6b44cddc59f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:02.818339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba865dbc-d4d1-4f7f-826c-6b44cddc59f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:02.819827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba865dbc-d4d1-4f7f-826c-6b44cddc59f5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:02.838717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 415/833] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:02.839486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba865dbc-d4d1-4f7f-826c-6b44cddc59f5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:02.849814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:02.868633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce6009-de83-4dd1-951a-fd1f18a72901 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:02.908473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7ebeb-9cc2-49d8-bb25-c347f184417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:02.909899 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7ebeb-9cc2-49d8-bb25-c347f184417f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:02.910415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7ebeb-9cc2-49d8-bb25-c347f184417f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:02.910911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7ebeb-9cc2-49d8-bb25-c347f184417f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:02.923270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7ebeb-9cc2-49d8-bb25-c347f184417f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5ea7ebeb-9cc2-49d8-bb25-c347f184417f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:02.931711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:02.932695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:31:02.933562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:02.937112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 419/834] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:31:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:02.948901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7ebeb-9cc2-49d8-bb25-c347f184417f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:02.953721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba865dbc-d4d1-4f7f-826c-6b44cddc59f5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:02.994165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fa33d-d4fd-4de3-a8d6-23ea30ce8925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:02.994835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fa33d-d4fd-4de3-a8d6-23ea30ce8925 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:02.995119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fa33d-d4fd-4de3-a8d6-23ea30ce8925 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:02.995394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fa33d-d4fd-4de3-a8d6-23ea30ce8925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:02.996376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fa33d-d4fd-4de3-a8d6-23ea30ce8925 None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=69ee145599384c6da2cba85a42832dcd) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:02.996688 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fa33d-d4fd-4de3-a8d6-23ea30ce8925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:02.998294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fa33d-d4fd-4de3-a8d6-23ea30ce8925 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:03.004967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 416/835] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:02 2022] GET /identity/v3/role_assignments?user.id=69ee145599384c6da2cba85a42832dcd&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=81457691f3a2456fa33358cb6a922d05 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:03.080763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7ebeb-9cc2-49d8-bb25-c347f184417f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:03.117543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:03.119847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ad182-7685-4bfa-acdd-0ec482c2e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:03.120489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad182-7685-4bfa-acdd-0ec482c2e44e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:03.121128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad182-7685-4bfa-acdd-0ec482c2e44e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:03.121518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad182-7685-4bfa-acdd-0ec482c2e44e None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:03.122423 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ad182-7685-4bfa-acdd-0ec482c2e44e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:03.122809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ad182-7685-4bfa-acdd-0ec482c2e44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:03.124307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ad182-7685-4bfa-acdd-0ec482c2e44e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:03.135273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 420/836] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:31:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:03.141436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fa33d-d4fd-4de3-a8d6-23ea30ce8925 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:03.217547 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:03.250447 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fa33d-d4fd-4de3-a8d6-23ea30ce8925 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:03.287735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23561cd5-d1c9-4a35-8b30-81ff8e819302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:03.288291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23561cd5-d1c9-4a35-8b30-81ff8e819302 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:03.288574 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23561cd5-d1c9-4a35-8b30-81ff8e819302 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:03.289022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23561cd5-d1c9-4a35-8b30-81ff8e819302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:03.302836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23561cd5-d1c9-4a35-8b30-81ff8e819302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:03.305191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23561cd5-d1c9-4a35-8b30-81ff8e819302 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-23561cd5-d1c9-4a35-8b30-81ff8e819302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:03.333168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:03.334129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:03.335035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:03.335035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:03.335035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:03.335035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:03.335035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 417/837] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:03.342992 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ad182-7685-4bfa-acdd-0ec482c2e44e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:03.457661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ad182-7685-4bfa-acdd-0ec482c2e44e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:03.493165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1d19f-ad3a-44f2-977f-0e818080df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:03.493837 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d19f-ad3a-44f2-977f-0e818080df46 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:03.494123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d19f-ad3a-44f2-977f-0e818080df46 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:03.494401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d19f-ad3a-44f2-977f-0e818080df46 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:03.495137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1d19f-ad3a-44f2-977f-0e818080df46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:03.495442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1d19f-ad3a-44f2-977f-0e818080df46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:03.497420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1d19f-ad3a-44f2-977f-0e818080df46 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:03.509962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 421/838] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:03.515730 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23561cd5-d1c9-4a35-8b30-81ff8e819302 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:03.627474 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23561cd5-d1c9-4a35-8b30-81ff8e819302 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:03.652335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:03.666225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597cfae-0475-445c-bfe7-67f5a0f877b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:03.667416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597cfae-0475-445c-bfe7-67f5a0f877b0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:03.667920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597cfae-0475-445c-bfe7-67f5a0f877b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:03.668959 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597cfae-0475-445c-bfe7-67f5a0f877b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:03.678182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597cfae-0475-445c-bfe7-67f5a0f877b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:03.680008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597cfae-0475-445c-bfe7-67f5a0f877b0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-7597cfae-0475-445c-bfe7-67f5a0f877b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:31:03.696231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:03.697292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:03.698239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:03.698239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:03.698239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:31:03.698239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:31:03.698239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:31:03.698239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:31:03.698239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:31:03.698239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:31:03.698239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 16:31:03.698239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:03.700702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 418/839] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:31:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:03.706165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1d19f-ad3a-44f2-977f-0e818080df46 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:03.755347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:03.817450 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1d19f-ad3a-44f2-977f-0e818080df46 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:03.854143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a527287-0d38-4cb5-8a34-7967057b3687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:03.854872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a527287-0d38-4cb5-8a34-7967057b3687 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:03.855282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a527287-0d38-4cb5-8a34-7967057b3687 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:03.855563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a527287-0d38-4cb5-8a34-7967057b3687 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:03.856437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a527287-0d38-4cb5-8a34-7967057b3687 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:03.856741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a527287-0d38-4cb5-8a34-7967057b3687 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:03.857003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3fecc-b23f-4c49-8e6b-2744cb24a412 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:03.857297 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3fecc-b23f-4c49-8e6b-2744cb24a412 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:03.857889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3fecc-b23f-4c49-8e6b-2744cb24a412 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:03.858211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a527287-0d38-4cb5-8a34-7967057b3687 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:03.859790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 419/840] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:03.865594 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12b31a-0b20-4267-bd84-5addca4f1fe7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:03.865888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12b31a-0b20-4267-bd84-5addca4f1fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:03.866204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12b31a-0b20-4267-bd84-5addca4f1fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:03.879850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 422/841] 10.222.0.88 () {56 vars in 1135 bytes} [Mon Jul 18 16:31:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:03.885181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a527287-0d38-4cb5-8a34-7967057b3687 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:03.889893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8b12b31a-0b20-4267-bd84-5addca4f1fe7 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:03.891099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8b12b31a-0b20-4267-bd84-5addca4f1fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:03.973359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 420/842] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:03.984570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9443d-9cb5-4ab2-80e4-49b01f3a31b7 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:03.984867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9443d-9cb5-4ab2-80e4-49b01f3a31b7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:03.985200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9443d-9cb5-4ab2-80e4-49b01f3a31b7 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:03.986813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 421/843] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:03.991921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3570c3d-1811-46f2-a5d4-7442dbb13f30 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:03.992476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3570c3d-1811-46f2-a5d4-7442dbb13f30 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:03.992754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3570c3d-1811-46f2-a5d4-7442dbb13f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:03.998372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a527287-0d38-4cb5-8a34-7967057b3687 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:03.999958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:04.016785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e3570c3d-1811-46f2-a5d4-7442dbb13f30 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:04.016785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e3570c3d-1811-46f2-a5d4-7442dbb13f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:04.038825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a03b9-d676-49e6-a6b4-b3132b82ada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:04.039479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a03b9-d676-49e6-a6b4-b3132b82ada9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:04.039760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a03b9-d676-49e6-a6b4-b3132b82ada9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:04.040040 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a03b9-d676-49e6-a6b4-b3132b82ada9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:04.087535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 422/844] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:04.088802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3570c3d-1811-46f2-a5d4-7442dbb13f30 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:04.129937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a03b9-d676-49e6-a6b4-b3132b82ada9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:04.131409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a03b9-d676-49e6-a6b4-b3132b82ada9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:04.138460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:04.203622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3570c3d-1811-46f2-a5d4-7442dbb13f30 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:04.238318 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 423/845] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:03 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:31:04.238318 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a03b9-d676-49e6-a6b4-b3132b82ada9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:04.238318 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:04.247513 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068056e1-980d-4822-a4d5-ae84ca9f8699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:04.264507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068056e1-980d-4822-a4d5-ae84ca9f8699 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:04.264507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068056e1-980d-4822-a4d5-ae84ca9f8699 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:04.264507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068056e1-980d-4822-a4d5-ae84ca9f8699 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:04.264507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068056e1-980d-4822-a4d5-ae84ca9f8699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-068056e1-980d-4822-a4d5-ae84ca9f8699 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:04.268587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:04.269780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 423/846] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:31:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:04.270996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068056e1-980d-4822-a4d5-ae84ca9f8699 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:04.350490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a03b9-d676-49e6-a6b4-b3132b82ada9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:04.385649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068056e1-980d-4822-a4d5-ae84ca9f8699 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:04.386420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f797d0e-9e61-4c75-bd67-0222ce3d5479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:04.387275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f797d0e-9e61-4c75-bd67-0222ce3d5479 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:04.387558 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f797d0e-9e61-4c75-bd67-0222ce3d5479 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:04.387832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f797d0e-9e61-4c75-bd67-0222ce3d5479 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:04.388477 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f797d0e-9e61-4c75-bd67-0222ce3d5479 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:04.388585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:04.389893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f797d0e-9e61-4c75-bd67-0222ce3d5479 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4f797d0e-9e61-4c75-bd67-0222ce3d5479 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:04.401776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:04.403643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:04.404473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 424/847] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:31:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:04.405697 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f797d0e-9e61-4c75-bd67-0222ce3d5479 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:04.431832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e6215-6bfb-435a-8435-43d386b76fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:04.433226 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e6215-6bfb-435a-8435-43d386b76fdd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:04.433826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e6215-6bfb-435a-8435-43d386b76fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:04.434114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e6215-6bfb-435a-8435-43d386b76fdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:04.448384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e6215-6bfb-435a-8435-43d386b76fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:04.449935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e6215-6bfb-435a-8435-43d386b76fdd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-8c3e6215-6bfb-435a-8435-43d386b76fdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:04.475094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:04.476055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:04.476998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:04.476998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:04.476998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:04.476998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:04.477225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:04.477624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 424/848] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:04.483482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e6215-6bfb-435a-8435-43d386b76fdd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:04.514840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f797d0e-9e61-4c75-bd67-0222ce3d5479 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:04.552385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57684cd5-7129-46a2-b034-96e8427c5060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:04.553027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684cd5-7129-46a2-b034-96e8427c5060 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:04.553313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684cd5-7129-46a2-b034-96e8427c5060 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:04.553589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57684cd5-7129-46a2-b034-96e8427c5060 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:04.554363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57684cd5-7129-46a2-b034-96e8427c5060 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:04.554722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57684cd5-7129-46a2-b034-96e8427c5060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:04.556123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57684cd5-7129-46a2-b034-96e8427c5060 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:04.568970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 425/849] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:31:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:04.568970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57684cd5-7129-46a2-b034-96e8427c5060 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:04.603287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e6215-6bfb-435a-8435-43d386b76fdd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:04.643021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc5f65-8cb7-4788-92c2-8737b8d34ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:04.643704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc5f65-8cb7-4788-92c2-8737b8d34ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:04.644089 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc5f65-8cb7-4788-92c2-8737b8d34ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:04.644370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc5f65-8cb7-4788-92c2-8737b8d34ac2 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:04.645127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc5f65-8cb7-4788-92c2-8737b8d34ac2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:04.645437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc5f65-8cb7-4788-92c2-8737b8d34ac2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:04.646821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc5f65-8cb7-4788-92c2-8737b8d34ac2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:04.656827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 425/850] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:31:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:04.656827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc5f65-8cb7-4788-92c2-8737b8d34ac2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:04.677014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:04.681557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57684cd5-7129-46a2-b034-96e8427c5060 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:04.714206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:04.723320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e76bfb-96c5-44ca-8076-863d37b826aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:04.724222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76bfb-96c5-44ca-8076-863d37b826aa None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:04.724532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76bfb-96c5-44ca-8076-863d37b826aa None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:04.724861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e76bfb-96c5-44ca-8076-863d37b826aa None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:04.725537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e76bfb-96c5-44ca-8076-863d37b826aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:04.725843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e76bfb-96c5-44ca-8076-863d37b826aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:04.727290 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e76bfb-96c5-44ca-8076-863d37b826aa None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:04.740619 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 426/851] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:04.742831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e76bfb-96c5-44ca-8076-863d37b826aa None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:04.774282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc5f65-8cb7-4788-92c2-8737b8d34ac2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:04.812238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f9fef-07aa-4e44-a4a3-1de8bb96ca44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:04.812993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f9fef-07aa-4e44-a4a3-1de8bb96ca44 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:04.813283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f9fef-07aa-4e44-a4a3-1de8bb96ca44 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:04.813561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f9fef-07aa-4e44-a4a3-1de8bb96ca44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:04.825107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f9fef-07aa-4e44-a4a3-1de8bb96ca44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:04.826681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f9fef-07aa-4e44-a4a3-1de8bb96ca44 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-880f9fef-07aa-4e44-a4a3-1de8bb96ca44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:04.846270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:04.847318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:04.848191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:04.848191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:04.848191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:04.848191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:04.852235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 426/852] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:04.853127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f9fef-07aa-4e44-a4a3-1de8bb96ca44 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:04.870462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e76bfb-96c5-44ca-8076-863d37b826aa None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:04.909643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378a27e-fc8b-4b7b-ba1f-9b204842676d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:04.910349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378a27e-fc8b-4b7b-ba1f-9b204842676d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:04.910635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378a27e-fc8b-4b7b-ba1f-9b204842676d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:04.910912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378a27e-fc8b-4b7b-ba1f-9b204842676d None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:04.911741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378a27e-fc8b-4b7b-ba1f-9b204842676d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:04.912074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378a27e-fc8b-4b7b-ba1f-9b204842676d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:04.913553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378a27e-fc8b-4b7b-ba1f-9b204842676d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:04.929020 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:04.929331 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 427/853] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:31:04 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:04.930731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378a27e-fc8b-4b7b-ba1f-9b204842676d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:04.976007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f9fef-07aa-4e44-a4a3-1de8bb96ca44 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:05.015616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:05.021541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d0c0a-75ca-4e5c-a72f-a56c91f94778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:05.022849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0c0a-75ca-4e5c-a72f-a56c91f94778 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:05.023348 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0c0a-75ca-4e5c-a72f-a56c91f94778 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:05.023812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d0c0a-75ca-4e5c-a72f-a56c91f94778 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:05.034171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d0c0a-75ca-4e5c-a72f-a56c91f94778 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:05.036344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d0c0a-75ca-4e5c-a72f-a56c91f94778 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a85d0c0a-75ca-4e5c-a72f-a56c91f94778 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:31:05.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:05.060587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:05.061542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:31:05.061542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:31:05.061542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:31:05.061542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:31:05.061542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:31:05.061542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:31:05.061542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:05.061542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:05.064308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 427/854] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:05.070249 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d0c0a-75ca-4e5c-a72f-a56c91f94778 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:05.082688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378a27e-fc8b-4b7b-ba1f-9b204842676d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:05.135488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405fc25-15b3-4f99-871f-0e8ea8428fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:05.136254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405fc25-15b3-4f99-871f-0e8ea8428fbb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:05.136552 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405fc25-15b3-4f99-871f-0e8ea8428fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:05.136842 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405fc25-15b3-4f99-871f-0e8ea8428fbb None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:05.138334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405fc25-15b3-4f99-871f-0e8ea8428fbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:05.138647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405fc25-15b3-4f99-871f-0e8ea8428fbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:05.140110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405fc25-15b3-4f99-871f-0e8ea8428fbb None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:05.153593 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 428/855] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:05.160428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405fc25-15b3-4f99-871f-0e8ea8428fbb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:05.218944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d0c0a-75ca-4e5c-a72f-a56c91f94778 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:05.253851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:05.259928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d90d4-3a7c-4409-90d1-a3ede4137bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:05.260603 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d90d4-3a7c-4409-90d1-a3ede4137bcb None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:05.260929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d90d4-3a7c-4409-90d1-a3ede4137bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:05.261201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d90d4-3a7c-4409-90d1-a3ede4137bcb None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:05.262679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d90d4-3a7c-4409-90d1-a3ede4137bcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:05.263013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d90d4-3a7c-4409-90d1-a3ede4137bcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:05.264473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d90d4-3a7c-4409-90d1-a3ede4137bcb None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:05.282519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405fc25-15b3-4f99-871f-0e8ea8428fbb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:05.282710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 428/856] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:31:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:05.284043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f71dd1-01d0-400c-a117-b263c6ad74b9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:05.284342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f71dd1-01d0-400c-a117-b263c6ad74b9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:05.284627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f71dd1-01d0-400c-a117-b263c6ad74b9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:05.289955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 429/857] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:05.289955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f71dd1-01d0-400c-a117-b263c6ad74b9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:05.298600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:05.321799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ef97c-f13e-4d5a-aee8-61b6e0242c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:05.322526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ef97c-f13e-4d5a-aee8-61b6e0242c20 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:05.322809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ef97c-f13e-4d5a-aee8-61b6e0242c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:05.323084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ef97c-f13e-4d5a-aee8-61b6e0242c20 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:05.331468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ef97c-f13e-4d5a-aee8-61b6e0242c20 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:05.333012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ef97c-f13e-4d5a-aee8-61b6e0242c20 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d70ef97c-f13e-4d5a-aee8-61b6e0242c20 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:31:05.349361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:05.350364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:05.351293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:05.351293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:05.351293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:31:05.351293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:31:05.351293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:31:05.351293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:31:05.351293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:31:05.351293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:31:05.351293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:31:05.351293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:05.351293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 429/858] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Jul 18 16:31:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:05.351867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b487064-dc3c-43c7-9e77-42fdddcbe072 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:05.352743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b487064-dc3c-43c7-9e77-42fdddcbe072 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:05.352743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b487064-dc3c-43c7-9e77-42fdddcbe072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:05.370405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7b487064-dc3c-43c7-9e77-42fdddcbe072 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:05.371656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7b487064-dc3c-43c7-9e77-42fdddcbe072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:05.406547 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f71dd1-01d0-400c-a117-b263c6ad74b9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:05.448554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6bbc2-3f58-42c0-a24b-5aab852f4df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:05.452949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6bbc2-3f58-42c0-a24b-5aab852f4df1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:05.453095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 430/859] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:05 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:05.453176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6bbc2-3f58-42c0-a24b-5aab852f4df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:05.453176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6bbc2-3f58-42c0-a24b-5aab852f4df1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:05.453324 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b487064-dc3c-43c7-9e77-42fdddcbe072 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:05.453398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6bbc2-3f58-42c0-a24b-5aab852f4df1 None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=bf341f1bb8944a3e9331678100b941a1) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:05.453398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6bbc2-3f58-42c0-a24b-5aab852f4df1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:05.453398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6bbc2-3f58-42c0-a24b-5aab852f4df1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:05.474463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 430/860] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:05 2022] GET /identity/v3/role_assignments?user.id=bf341f1bb8944a3e9331678100b941a1&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=81457691f3a2456fa33358cb6a922d05 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:05.474463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6bbc2-3f58-42c0-a24b-5aab852f4df1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:05.561263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:05.570070 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b487064-dc3c-43c7-9e77-42fdddcbe072 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:05.571756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:05.596584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6bbc2-3f58-42c0-a24b-5aab852f4df1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:05.610748 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdc32c-732b-462c-8119-1adf5a290c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:05.611502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdc32c-732b-462c-8119-1adf5a290c97 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:05.611785 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdc32c-732b-462c-8119-1adf5a290c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:05.612059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdc32c-732b-462c-8119-1adf5a290c97 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:05.612935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdc32c-732b-462c-8119-1adf5a290c97 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:05.618569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdc32c-732b-462c-8119-1adf5a290c97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:05.618569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdc32c-732b-462c-8119-1adf5a290c97 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:05.637048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddd38c-1bf4-4886-b1af-c29a98ff4f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:05.637970 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddd38c-1bf4-4886-b1af-c29a98ff4f9c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:05.638310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddd38c-1bf4-4886-b1af-c29a98ff4f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:05.638696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddd38c-1bf4-4886-b1af-c29a98ff4f9c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:05.639745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddd38c-1bf4-4886-b1af-c29a98ff4f9c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:05.641236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddd38c-1bf4-4886-b1af-c29a98ff4f9c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:05.646762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 431/861] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Jul 18 16:31:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:05.651919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdc32c-732b-462c-8119-1adf5a290c97 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2bddd38c-1bf4-4886-b1af-c29a98ff4f9c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:05.655236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:05.656273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:31:05.657152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:05.660985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 431/862] 10.222.0.88 () {56 vars in 1132 bytes} [Mon Jul 18 16:31:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:05.662619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddd38c-1bf4-4886-b1af-c29a98ff4f9c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:05.770973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdc32c-732b-462c-8119-1adf5a290c97 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:05.779364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddd38c-1bf4-4886-b1af-c29a98ff4f9c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:05.798671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:05.818351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d34991-6f5f-4b07-9ddf-5d2ed0c50eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:05.818943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d52762-39c0-4428-ade0-b9bc098a8ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:05.819101 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d34991-6f5f-4b07-9ddf-5d2ed0c50eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:05.819359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d34991-6f5f-4b07-9ddf-5d2ed0c50eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:05.819690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d34991-6f5f-4b07-9ddf-5d2ed0c50eb6 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:05.820205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d52762-39c0-4428-ade0-b9bc098a8ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:05.820405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d34991-6f5f-4b07-9ddf-5d2ed0c50eb6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:05.820767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d34991-6f5f-4b07-9ddf-5d2ed0c50eb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:05.820931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d52762-39c0-4428-ade0-b9bc098a8ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:05.821350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d52762-39c0-4428-ade0-b9bc098a8ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:05.822365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d34991-6f5f-4b07-9ddf-5d2ed0c50eb6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:05.832045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 432/863] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Jul 18 16:31:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:05.833954 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d34991-6f5f-4b07-9ddf-5d2ed0c50eb6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:05.866771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:05.923346 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d52762-39c0-4428-ade0-b9bc098a8ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:05.925618 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d52762-39c0-4428-ade0-b9bc098a8ee4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:05.960899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d34991-6f5f-4b07-9ddf-5d2ed0c50eb6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:06.000148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf50f1-ffe7-48c4-ac8f-41b9b19c68e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:06.000981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf50f1-ffe7-48c4-ac8f-41b9b19c68e1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:06.001274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf50f1-ffe7-48c4-ac8f-41b9b19c68e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:06.001561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf50f1-ffe7-48c4-ac8f-41b9b19c68e1 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:06.002311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf50f1-ffe7-48c4-ac8f-41b9b19c68e1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:06.002581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf50f1-ffe7-48c4-ac8f-41b9b19c68e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:06.004080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf50f1-ffe7-48c4-ac8f-41b9b19c68e1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:06.013750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 433/864] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Jul 18 16:31:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:06.055770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 432/865] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:05 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:31:06.061618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf50f1-ffe7-48c4-ac8f-41b9b19c68e1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:06.117680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:06.179724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf50f1-ffe7-48c4-ac8f-41b9b19c68e1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:06.219274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e34f49-64f4-415f-8d91-3c5a2193af9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:06.220076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e34f49-64f4-415f-8d91-3c5a2193af9c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:06.220364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e34f49-64f4-415f-8d91-3c5a2193af9c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:06.220638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e34f49-64f4-415f-8d91-3c5a2193af9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:06.233001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e34f49-64f4-415f-8d91-3c5a2193af9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:06.234456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e34f49-64f4-415f-8d91-3c5a2193af9c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-81e34f49-64f4-415f-8d91-3c5a2193af9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:06.253379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:06.254416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:06.255304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:06.255304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:06.255304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:06.255304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:06.255304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 434/866] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:06.262188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d52762-39c0-4428-ade0-b9bc098a8ee4 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:06.329963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:06.372227 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d52762-39c0-4428-ade0-b9bc098a8ee4 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:06.408180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c750f7-09b6-4579-8998-73d58feed40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:06.408843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c750f7-09b6-4579-8998-73d58feed40e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:06.409151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c750f7-09b6-4579-8998-73d58feed40e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:06.409425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c750f7-09b6-4579-8998-73d58feed40e None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:06.410169 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c750f7-09b6-4579-8998-73d58feed40e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:06.410490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c750f7-09b6-4579-8998-73d58feed40e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:06.411860 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c750f7-09b6-4579-8998-73d58feed40e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:06.424960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 433/867] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:06.434184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e34f49-64f4-415f-8d91-3c5a2193af9c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:06.539694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:06.544669 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e34f49-64f4-415f-8d91-3c5a2193af9c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:06.585675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d7c43-4b5f-4f10-b252-ca88661da580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:06.586476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7c43-4b5f-4f10-b252-ca88661da580 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:06.586759 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7c43-4b5f-4f10-b252-ca88661da580 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:06.587032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d7c43-4b5f-4f10-b252-ca88661da580 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:06.595195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d7c43-4b5f-4f10-b252-ca88661da580 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:06.596644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d7c43-4b5f-4f10-b252-ca88661da580 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3f1d7c43-4b5f-4f10-b252-ca88661da580 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:31:06.611514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:06.612444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:06.613364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:31:06.613364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:31:06.613364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:31:06.613364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:31:06.613364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:31:06.613364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:31:06.613364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:06.613364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:06.613364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 435/868] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:06.620321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c750f7-09b6-4579-8998-73d58feed40e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:06.729165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c750f7-09b6-4579-8998-73d58feed40e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:06.750746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:06.770436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6fc03-7260-4a27-9b10-785cc33684bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:06.771104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6fc03-7260-4a27-9b10-785cc33684bc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:06.771390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6fc03-7260-4a27-9b10-785cc33684bc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:06.771668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6fc03-7260-4a27-9b10-785cc33684bc None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:06.772467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6fc03-7260-4a27-9b10-785cc33684bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:06.772777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6fc03-7260-4a27-9b10-785cc33684bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:06.774255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6fc03-7260-4a27-9b10-785cc33684bc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:06.787614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 434/869] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:31:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:06.793572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d7c43-4b5f-4f10-b252-ca88661da580 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:06.903493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d7c43-4b5f-4f10-b252-ca88661da580 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:06.940282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f0e36-066f-4084-87fb-98e0fa35c25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:06.940282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f0e36-066f-4084-87fb-98e0fa35c25c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:06.940606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f0e36-066f-4084-87fb-98e0fa35c25c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:06.944906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f0e36-066f-4084-87fb-98e0fa35c25c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:06.944906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f0e36-066f-4084-87fb-98e0fa35c25c None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=3f38abef63e24567a995b30c144177fa) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:06.944906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f0e36-066f-4084-87fb-98e0fa35c25c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:06.944906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f0e36-066f-4084-87fb-98e0fa35c25c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:06.951474 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 436/870] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:06 2022] GET /identity/v3/role_assignments?user.id=3f38abef63e24567a995b30c144177fa&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:07.096791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c403423-a674-467d-accc-f66dace2c6f0 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:07.097230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c403423-a674-467d-accc-f66dace2c6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:07.097429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c403423-a674-467d-accc-f66dace2c6f0 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:07.099282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 435/871] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:07.105227 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4bc1d-6ce6-4c9f-949e-b00c05d21eaa None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:07.106150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4bc1d-6ce6-4c9f-949e-b00c05d21eaa None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:07.106480 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4bc1d-6ce6-4c9f-949e-b00c05d21eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:07.126060 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-16b4bc1d-6ce6-4c9f-949e-b00c05d21eaa None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:07.127277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-16b4bc1d-6ce6-4c9f-949e-b00c05d21eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:07.147554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:07.212976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 437/872] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:07.237841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c403423-a674-467d-accc-f66dace2c6f0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:07.360011 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c403423-a674-467d-accc-f66dace2c6f0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:07.400468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0cf89-906d-4578-9192-59d9ad983dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:07.401233 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0cf89-906d-4578-9192-59d9ad983dcd None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:07.401523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0cf89-906d-4578-9192-59d9ad983dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:07.401798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0cf89-906d-4578-9192-59d9ad983dcd None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:07.402554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0cf89-906d-4578-9192-59d9ad983dcd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:07.403928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0cf89-906d-4578-9192-59d9ad983dcd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:07.493240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 436/873] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:31:07 2022] POST /identity/v3/services => generated 251 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:07.619856 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc13885-380f-4367-b31b-4facdf77537c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:07.620210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc13885-380f-4367-b31b-4facdf77537c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:07.620393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc13885-380f-4367-b31b-4facdf77537c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:07.622678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 438/874] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:07.629396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3ddac-2be3-495b-9c44-45029a64e79e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:07.629898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3ddac-2be3-495b-9c44-45029a64e79e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:07.630421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3ddac-2be3-495b-9c44-45029a64e79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:07.662485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c4a3ddac-2be3-495b-9c44-45029a64e79e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:07.664594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c4a3ddac-2be3-495b-9c44-45029a64e79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:07.729648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:07.801286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 437/875] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:07 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:07.823246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc13885-380f-4367-b31b-4facdf77537c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:07.950718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc13885-380f-4367-b31b-4facdf77537c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:07.987625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efe456-3f1b-4b39-9456-fad6413f3b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:07.988499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efe456-3f1b-4b39-9456-fad6413f3b21 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:07.988788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efe456-3f1b-4b39-9456-fad6413f3b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:07.989098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efe456-3f1b-4b39-9456-fad6413f3b21 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:07.989756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efe456-3f1b-4b39-9456-fad6413f3b21 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:07.991142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efe456-3f1b-4b39-9456-fad6413f3b21 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-03efe456-3f1b-4b39-9456-fad6413f3b21 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:08.005046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:08.006167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:08.007177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 439/876] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Jul 18 16:31:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:08.013021 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3ddac-2be3-495b-9c44-45029a64e79e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:08.128891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3ddac-2be3-495b-9c44-45029a64e79e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:08.168994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea96aa3-b224-4f93-b07f-9fe27faf7eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:08.169678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea96aa3-b224-4f93-b07f-9fe27faf7eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:08.170012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea96aa3-b224-4f93-b07f-9fe27faf7eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:08.170297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea96aa3-b224-4f93-b07f-9fe27faf7eb9 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:08.171012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea96aa3-b224-4f93-b07f-9fe27faf7eb9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:08.171320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea96aa3-b224-4f93-b07f-9fe27faf7eb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:08.172726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea96aa3-b224-4f93-b07f-9fe27faf7eb9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:08.178363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 438/877] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Jul 18 16:31:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:08.186065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efe456-3f1b-4b39-9456-fad6413f3b21 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:08.206770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:08.211642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffd0fc-645b-45ed-861c-d06c7a9639b6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:08.211927 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffd0fc-645b-45ed-861c-d06c7a9639b6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:08.222721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffd0fc-645b-45ed-861c-d06c7a9639b6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:08.222721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 439/878] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:08.222721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319cd57-3598-434e-8cbe-d689340d79ad None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:08.222721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319cd57-3598-434e-8cbe-d689340d79ad None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:08.222721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319cd57-3598-434e-8cbe-d689340d79ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:08.241466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4319cd57-3598-434e-8cbe-d689340d79ad None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:08.243303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4319cd57-3598-434e-8cbe-d689340d79ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:08.289670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:08.313630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efe456-3f1b-4b39-9456-fad6413f3b21 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:08.356436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 440/879] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:08 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:08.360455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fda9be-b81f-4013-af51-c8b466185f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:08.361344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fda9be-b81f-4013-af51-c8b466185f34 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:08.361790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fda9be-b81f-4013-af51-c8b466185f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:08.362219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fda9be-b81f-4013-af51-c8b466185f34 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:08.363173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fda9be-b81f-4013-af51-c8b466185f34 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:08.363704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fda9be-b81f-4013-af51-c8b466185f34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:08.365393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fda9be-b81f-4013-af51-c8b466185f34 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:08.371340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 440/880] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Jul 18 16:31:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:08.385755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319cd57-3598-434e-8cbe-d689340d79ad None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:08.499570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319cd57-3598-434e-8cbe-d689340d79ad None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:08.536306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6ae58-9c9c-4044-9a34-4887e54b4776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:08.537298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6ae58-9c9c-4044-9a34-4887e54b4776 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:08.537723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6ae58-9c9c-4044-9a34-4887e54b4776 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:08.538138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6ae58-9c9c-4044-9a34-4887e54b4776 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:08.539019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6ae58-9c9c-4044-9a34-4887e54b4776 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:08.540539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6ae58-9c9c-4044-9a34-4887e54b4776 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:08.666407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 441/881] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:31:08 2022] POST /identity/v3/services => generated 269 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:09.255245 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7db115-7e29-4a51-95a7-81117355ce53 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:09.255752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7db115-7e29-4a51-95a7-81117355ce53 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:09.255848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7db115-7e29-4a51-95a7-81117355ce53 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:09.257508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 441/882] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:09.263467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2db84-0066-445d-a0cc-707e00d19b88 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:09.263768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2db84-0066-445d-a0cc-707e00d19b88 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:09.264045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2db84-0066-445d-a0cc-707e00d19b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:09.282011 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3fc2db84-0066-445d-a0cc-707e00d19b88 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:09.283205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3fc2db84-0066-445d-a0cc-707e00d19b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:09.298970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:09.361312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 442/883] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:09.384550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7db115-7e29-4a51-95a7-81117355ce53 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:09.470696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:09.491548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7db115-7e29-4a51-95a7-81117355ce53 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:09.530693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc465f8e-9ead-494a-b3e9-51a12f4480d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:09.531850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc465f8e-9ead-494a-b3e9-51a12f4480d7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:09.532329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc465f8e-9ead-494a-b3e9-51a12f4480d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:09.532844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc465f8e-9ead-494a-b3e9-51a12f4480d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:09.545281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc465f8e-9ead-494a-b3e9-51a12f4480d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-dc465f8e-9ead-494a-b3e9-51a12f4480d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:09.554199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:09.555184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:09.556096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:09.559045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 442/884] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Jul 18 16:31:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:09.564943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2db84-0066-445d-a0cc-707e00d19b88 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:09.663231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736f5f-6628-4701-9cec-d0456a6ee5ed None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:09.663601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736f5f-6628-4701-9cec-d0456a6ee5ed None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:09.663801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80736f5f-6628-4701-9cec-d0456a6ee5ed None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:09.666448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 443/885] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:09.671040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ce023-a07c-4c13-bdc1-295ad05325b9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:09.671329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ce023-a07c-4c13-bdc1-295ad05325b9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:09.671780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ce023-a07c-4c13-bdc1-295ad05325b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:09.677681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2db84-0066-445d-a0cc-707e00d19b88 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:09.689264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-482ce023-a07c-4c13-bdc1-295ad05325b9 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:09.690745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-482ce023-a07c-4c13-bdc1-295ad05325b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:09.718829 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff58552-0975-46b7-926d-67eadde92944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:09.719988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff58552-0975-46b7-926d-67eadde92944 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:09.720504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff58552-0975-46b7-926d-67eadde92944 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:09.721011 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff58552-0975-46b7-926d-67eadde92944 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:09.721948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff58552-0975-46b7-926d-67eadde92944 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:09.722453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff58552-0975-46b7-926d-67eadde92944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:09.724033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff58552-0975-46b7-926d-67eadde92944 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:09.733906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 443/886] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Jul 18 16:31:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:09.739056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff58552-0975-46b7-926d-67eadde92944 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:09.752325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:09.776581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 444/887] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:09 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:09.800175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ce023-a07c-4c13-bdc1-295ad05325b9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:09.861614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff58552-0975-46b7-926d-67eadde92944 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:09.876620 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:09.899614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8f222-7596-457b-8679-22428f048554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:09.900709 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8f222-7596-457b-8679-22428f048554 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:09.901169 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8f222-7596-457b-8679-22428f048554 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:09.901691 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8f222-7596-457b-8679-22428f048554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:09.912172 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ce023-a07c-4c13-bdc1-295ad05325b9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:09.915845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8f222-7596-457b-8679-22428f048554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:09.917516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8f222-7596-457b-8679-22428f048554 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6ee8f222-7596-457b-8679-22428f048554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:09.938061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:09.939239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:09.940351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:09.940351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:09.940351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:09.940351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:09.942470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 444/888] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:09.948282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8f222-7596-457b-8679-22428f048554 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:09.958281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a1a56-6925-4c74-9730-b330fec18a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:09.959086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a1a56-6925-4c74-9730-b330fec18a09 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:09.959372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a1a56-6925-4c74-9730-b330fec18a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:09.959652 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a1a56-6925-4c74-9730-b330fec18a09 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:09.960312 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a1a56-6925-4c74-9730-b330fec18a09 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:09.961846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a1a56-6925-4c74-9730-b330fec18a09 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4c9a1a56-6925-4c74-9730-b330fec18a09 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:09.975253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:09.976281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:09.977161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 445/889] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:31:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:09.983081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a1a56-6925-4c74-9730-b330fec18a09 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:10.030906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:10.084507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8f222-7596-457b-8679-22428f048554 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:10.118770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a1a56-6925-4c74-9730-b330fec18a09 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:10.126344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03248489-be23-40e8-80bc-2952c36b5d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:10.127364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03248489-be23-40e8-80bc-2952c36b5d96 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.127858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03248489-be23-40e8-80bc-2952c36b5d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.128433 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03248489-be23-40e8-80bc-2952c36b5d96 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.129490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03248489-be23-40e8-80bc-2952c36b5d96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:10.130108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03248489-be23-40e8-80bc-2952c36b5d96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:10.131715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03248489-be23-40e8-80bc-2952c36b5d96 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:10.146729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 445/890] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:10.151947 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03248489-be23-40e8-80bc-2952c36b5d96 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:10.163887 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fe2bb-4aef-4480-95e6-d287eb7aa6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:10.164617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe2bb-4aef-4480-95e6-d287eb7aa6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.164926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe2bb-4aef-4480-95e6-d287eb7aa6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.165213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fe2bb-4aef-4480-95e6-d287eb7aa6d7 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.165927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fe2bb-4aef-4480-95e6-d287eb7aa6d7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:10.166235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fe2bb-4aef-4480-95e6-d287eb7aa6d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:10.167611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55fe2bb-4aef-4480-95e6-d287eb7aa6d7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:10.172339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:10.177228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 446/891] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:31:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:10.184994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fe2bb-4aef-4480-95e6-d287eb7aa6d7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:10.271468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03248489-be23-40e8-80bc-2952c36b5d96 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:10.295255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:10.312028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa16150-a98a-4ca7-8f24-1c91f634a009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:10.315840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa16150-a98a-4ca7-8f24-1c91f634a009 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.316166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa16150-a98a-4ca7-8f24-1c91f634a009 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.316454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa16150-a98a-4ca7-8f24-1c91f634a009 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.325847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa16150-a98a-4ca7-8f24-1c91f634a009 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:10.327366 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fe2bb-4aef-4480-95e6-d287eb7aa6d7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:10.328476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa16150-a98a-4ca7-8f24-1c91f634a009 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5fa16150-a98a-4ca7-8f24-1c91f634a009 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:31:10.349250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:10.350251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:10.351222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:10.351222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:10.351222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:31:10.351222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:31:10.351222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:31:10.351222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:31:10.351222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:31:10.351222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:31:10.351222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:31:10.351222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:10.359200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 446/892] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Jul 18 16:31:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:10.360500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa16150-a98a-4ca7-8f24-1c91f634a009 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:10.375142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a93bdf-3ee6-4090-988c-5d2d02cc120e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:10.376073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a93bdf-3ee6-4090-988c-5d2d02cc120e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.376269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a93bdf-3ee6-4090-988c-5d2d02cc120e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.376550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a93bdf-3ee6-4090-988c-5d2d02cc120e None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.377376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a93bdf-3ee6-4090-988c-5d2d02cc120e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:10.377684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a93bdf-3ee6-4090-988c-5d2d02cc120e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:10.379227 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a93bdf-3ee6-4090-988c-5d2d02cc120e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:10.390772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 447/893] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:31:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:10.392382 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a93bdf-3ee6-4090-988c-5d2d02cc120e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:10.469764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:10.490633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa16150-a98a-4ca7-8f24-1c91f634a009 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:10.515041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a93bdf-3ee6-4090-988c-5d2d02cc120e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:10.531202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104e2e6-d20d-4241-ba9e-a9dc51cae360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:10.532265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104e2e6-d20d-4241-ba9e-a9dc51cae360 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.532701 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104e2e6-d20d-4241-ba9e-a9dc51cae360 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.533027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104e2e6-d20d-4241-ba9e-a9dc51cae360 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.533879 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104e2e6-d20d-4241-ba9e-a9dc51cae360 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:10.534190 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104e2e6-d20d-4241-ba9e-a9dc51cae360 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:10.535636 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104e2e6-d20d-4241-ba9e-a9dc51cae360 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:10.553495 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 447/894] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Jul 18 16:31:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:10.554611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2076bf1-7518-4872-9d6d-2705a8270e21 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.554901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2076bf1-7518-4872-9d6d-2705a8270e21 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.555175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2076bf1-7518-4872-9d6d-2705a8270e21 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.556552 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 448/895] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:10.558444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44335505-3513-438b-826f-c2ae9c664ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:10.559166 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44335505-3513-438b-826f-c2ae9c664ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.559515 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44335505-3513-438b-826f-c2ae9c664ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.559800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44335505-3513-438b-826f-c2ae9c664ad8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.560737 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44335505-3513-438b-826f-c2ae9c664ad8 None admin] RBAC: Adding query filter params (interface=public, service_id=62e04ae59542410cb7b9acbadc4c0a35, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:10.561083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44335505-3513-438b-826f-c2ae9c664ad8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:10.561246 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2076bf1-7518-4872-9d6d-2705a8270e21 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:10.562514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44335505-3513-438b-826f-c2ae9c664ad8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:10.572944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 448/896] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Jul 18 16:31:10 2022] GET /identity/v3/endpoints?service_id=62e04ae59542410cb7b9acbadc4c0a35&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:10.572944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab18c6-869e-4f22-a59e-cdf6c37122e3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.572944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab18c6-869e-4f22-a59e-cdf6c37122e3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.572944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab18c6-869e-4f22-a59e-cdf6c37122e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.591198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4eab18c6-869e-4f22-a59e-cdf6c37122e3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:10.592408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4eab18c6-869e-4f22-a59e-cdf6c37122e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:10.607306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:10.680910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 449/897] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:10.680910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab18c6-869e-4f22-a59e-cdf6c37122e3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:10.683845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2076bf1-7518-4872-9d6d-2705a8270e21 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:10.723190 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977f8b5-07d2-4ad5-a234-86d257c683a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:10.723939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977f8b5-07d2-4ad5-a234-86d257c683a8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.724120 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977f8b5-07d2-4ad5-a234-86d257c683a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.724397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977f8b5-07d2-4ad5-a234-86d257c683a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.732468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:10.781011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab18c6-869e-4f22-a59e-cdf6c37122e3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:10.800651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977f8b5-07d2-4ad5-a234-86d257c683a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:10.802163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977f8b5-07d2-4ad5-a234-86d257c683a8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:10.821479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11a437-4a30-4ff6-afdd-abcd496fc9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:10.822533 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a437-4a30-4ff6-afdd-abcd496fc9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.822984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a437-4a30-4ff6-afdd-abcd496fc9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.823455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec11a437-4a30-4ff6-afdd-abcd496fc9b4 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.824851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11a437-4a30-4ff6-afdd-abcd496fc9b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:10.826521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec11a437-4a30-4ff6-afdd-abcd496fc9b4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:10.833403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 450/898] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Jul 18 16:31:10 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:31:10.834466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11a437-4a30-4ff6-afdd-abcd496fc9b4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:10.852647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:10.941298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 449/899] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:10 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:31:10.942871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2591d7-758a-4aac-b5c8-11ae193b7a0b None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:10.943165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2591d7-758a-4aac-b5c8-11ae193b7a0b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:10.943447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2591d7-758a-4aac-b5c8-11ae193b7a0b None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:10.945183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 450/900] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:10.945946 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2591d7-758a-4aac-b5c8-11ae193b7a0b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:10.974189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec11a437-4a30-4ff6-afdd-abcd496fc9b4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:11.001004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:11.018847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1cb7b-b461-4998-ad3d-48898dc209bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:11.019185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1cb7b-b461-4998-ad3d-48898dc209bf None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:11.019546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1cb7b-b461-4998-ad3d-48898dc209bf None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:11.019757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1cb7b-b461-4998-ad3d-48898dc209bf None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:11.020498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1cb7b-b461-4998-ad3d-48898dc209bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:11.021924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1cb7b-b461-4998-ad3d-48898dc209bf None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:11.063698 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2591d7-758a-4aac-b5c8-11ae193b7a0b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:11.101290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 451/901] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:31:10 2022] POST /identity/v3/services => generated 241 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:11.102875 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785856d-e9ce-43ff-ad3f-c2377bd76a0b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:11.103077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785856d-e9ce-43ff-ad3f-c2377bd76a0b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:11.103357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785856d-e9ce-43ff-ad3f-c2377bd76a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:11.110876 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b363f50-55e7-438d-833e-ffd246c871c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:11.112093 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b363f50-55e7-438d-833e-ffd246c871c9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:11.112521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b363f50-55e7-438d-833e-ffd246c871c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:11.112987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b363f50-55e7-438d-833e-ffd246c871c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:11.116474 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:11.135133 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b363f50-55e7-438d-833e-ffd246c871c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:11.137549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b363f50-55e7-438d-833e-ffd246c871c9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:11.138723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d785856d-e9ce-43ff-ad3f-c2377bd76a0b None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:11.141035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d785856d-e9ce-43ff-ad3f-c2377bd76a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9b363f50-55e7-438d-833e-ffd246c871c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:11.169769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:11.170742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:11.172399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:11.172399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:11.172399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:11.172399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:11.172399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 451/902] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:11.178035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b363f50-55e7-438d-833e-ffd246c871c9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:11.232790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 452/903] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:11.257120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785856d-e9ce-43ff-ad3f-c2377bd76a0b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:11.314855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b363f50-55e7-438d-833e-ffd246c871c9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:11.353717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bd2f1-2721-4f82-a77f-bae43eeee33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:11.354385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bd2f1-2721-4f82-a77f-bae43eeee33a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:11.354669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bd2f1-2721-4f82-a77f-bae43eeee33a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:11.354948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bd2f1-2721-4f82-a77f-bae43eeee33a None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:11.356034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bd2f1-2721-4f82-a77f-bae43eeee33a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:11.356483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bd2f1-2721-4f82-a77f-bae43eeee33a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:11.357985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486bd2f1-2721-4f82-a77f-bae43eeee33a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:11.374289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 452/904] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:11.377112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bd2f1-2721-4f82-a77f-bae43eeee33a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:11.384034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785856d-e9ce-43ff-ad3f-c2377bd76a0b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:11.400179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:11.432827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa011185-0f69-40b9-999b-67af014e8c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:11.434486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa011185-0f69-40b9-999b-67af014e8c08 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:11.435495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa011185-0f69-40b9-999b-67af014e8c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:11.437511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa011185-0f69-40b9-999b-67af014e8c08 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:11.438746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:11.440128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa011185-0f69-40b9-999b-67af014e8c08 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:11.441967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa011185-0f69-40b9-999b-67af014e8c08 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-aa011185-0f69-40b9-999b-67af014e8c08 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:11.459279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:11.460295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:31:11.461184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:11.465057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 453/905] 10.222.0.88 () {56 vars in 1132 bytes} [Mon Jul 18 16:31:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:11.472094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa011185-0f69-40b9-999b-67af014e8c08 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:11.512489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bd2f1-2721-4f82-a77f-bae43eeee33a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:11.551569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6708da-5e23-4e69-9c32-132584c67491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:11.552650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6708da-5e23-4e69-9c32-132584c67491 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:11.552979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6708da-5e23-4e69-9c32-132584c67491 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:11.553838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6708da-5e23-4e69-9c32-132584c67491 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:11.562519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6708da-5e23-4e69-9c32-132584c67491 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:11.564204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6708da-5e23-4e69-9c32-132584c67491 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-cf6708da-5e23-4e69-9c32-132584c67491 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:31:11.582721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:11.583678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:11.584555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:31:11.584555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:31:11.584555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:31:11.584555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:31:11.584555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:31:11.584555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:31:11.584555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:11.584555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:11.588858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 453/906] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:11.589936 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6708da-5e23-4e69-9c32-132584c67491 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:11.622015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa011185-0f69-40b9-999b-67af014e8c08 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:11.668054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c33d34-b437-4818-85ea-35103016a632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:11.669743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c33d34-b437-4818-85ea-35103016a632 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:11.670935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c33d34-b437-4818-85ea-35103016a632 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:11.671367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c33d34-b437-4818-85ea-35103016a632 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:11.672226 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c33d34-b437-4818-85ea-35103016a632 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:11.672711 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c33d34-b437-4818-85ea-35103016a632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:11.674277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c33d34-b437-4818-85ea-35103016a632 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:11.685548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 454/907] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Jul 18 16:31:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:11.687290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c33d34-b437-4818-85ea-35103016a632 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:11.706653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6708da-5e23-4e69-9c32-132584c67491 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:11.715529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:11.741164 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:11.747441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f6de0-8690-48be-990a-85518d094984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:11.748357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f6de0-8690-48be-990a-85518d094984 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:11.748852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f6de0-8690-48be-990a-85518d094984 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:11.749393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f6de0-8690-48be-990a-85518d094984 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:11.750406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f6de0-8690-48be-990a-85518d094984 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:11.750937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f6de0-8690-48be-990a-85518d094984 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:11.752593 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f6de0-8690-48be-990a-85518d094984 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:11.774097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 454/908] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:31:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:11.775292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f6de0-8690-48be-990a-85518d094984 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:11.818214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c33d34-b437-4818-85ea-35103016a632 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:11.863884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b3679-35a8-4976-9f13-b16b3e1ca0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:11.864973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b3679-35a8-4976-9f13-b16b3e1ca0dc None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:11.865466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b3679-35a8-4976-9f13-b16b3e1ca0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:11.865955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b3679-35a8-4976-9f13-b16b3e1ca0dc None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:11.867173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b3679-35a8-4976-9f13-b16b3e1ca0dc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:11.867790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b3679-35a8-4976-9f13-b16b3e1ca0dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:11.869538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b3679-35a8-4976-9f13-b16b3e1ca0dc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:11.876128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 455/909] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Jul 18 16:31:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:11.882612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b3679-35a8-4976-9f13-b16b3e1ca0dc None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:11.911464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f6de0-8690-48be-990a-85518d094984 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:11.977363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c6557-2f03-41db-9a22-257520ab6aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:11.978701 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c6557-2f03-41db-9a22-257520ab6aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:11.979319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c6557-2f03-41db-9a22-257520ab6aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:11.979940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c6557-2f03-41db-9a22-257520ab6aa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:11.981928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c6557-2f03-41db-9a22-257520ab6aa8 None admin] RBAC: Adding query filter params (role.id=443cff586282466c87ffce62f1440b63, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=3f38abef63e24567a995b30c144177fa) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:11.982402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c6557-2f03-41db-9a22-257520ab6aa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:11.984016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c6557-2f03-41db-9a22-257520ab6aa8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:11.991124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 455/910] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:11 2022] GET /identity/v3/role_assignments?user.id=3f38abef63e24567a995b30c144177fa&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=443cff586282466c87ffce62f1440b63 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:12.023679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b3679-35a8-4976-9f13-b16b3e1ca0dc None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:12.025613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:12.066108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f6c29-d3d8-4871-a7eb-2b802d018061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:12.066861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f6c29-d3d8-4871-a7eb-2b802d018061 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:12.067177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f6c29-d3d8-4871-a7eb-2b802d018061 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:12.067468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f6c29-d3d8-4871-a7eb-2b802d018061 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:12.068305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f6c29-d3d8-4871-a7eb-2b802d018061 None admin] RBAC: Adding query filter params (interface=public, service_id=3258764919124f2ab020b42b23e413dc, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:12.068622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f6c29-d3d8-4871-a7eb-2b802d018061 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:12.070088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f6c29-d3d8-4871-a7eb-2b802d018061 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:12.077414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 456/911] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Jul 18 16:31:11 2022] GET /identity/v3/endpoints?service_id=3258764919124f2ab020b42b23e413dc&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:12.083042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c6557-2f03-41db-9a22-257520ab6aa8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:12.120002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:12.198036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c6557-2f03-41db-9a22-257520ab6aa8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:12.248445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb00217-bb54-441c-8636-ef05a0d16a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:12.249518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb00217-bb54-441c-8636-ef05a0d16a8b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:12.250027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb00217-bb54-441c-8636-ef05a0d16a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:12.250495 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb00217-bb54-441c-8636-ef05a0d16a8b None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:12.251374 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb00217-bb54-441c-8636-ef05a0d16a8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:12.252969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb00217-bb54-441c-8636-ef05a0d16a8b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:12.260427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 456/912] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Jul 18 16:31:12 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:31:12.977230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b26341-059d-404d-b064-2919615a600b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:12.977735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b26341-059d-404d-b064-2919615a600b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:12.977812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b26341-059d-404d-b064-2919615a600b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:12.979461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 457/913] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:12.985502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b7bd5-a487-49f6-8595-22a2bda062fa None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:12.986404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b7bd5-a487-49f6-8595-22a2bda062fa None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:12.986896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b7bd5-a487-49f6-8595-22a2bda062fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:13.008389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c64b7bd5-a487-49f6-8595-22a2bda062fa None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:13.009765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c64b7bd5-a487-49f6-8595-22a2bda062fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:13.082066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 457/914] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:13.104285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b26341-059d-404d-b064-2919615a600b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:13.218772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b26341-059d-404d-b064-2919615a600b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:13.256234 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed79a8-d8e9-447a-95dc-da3a00a061e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:13.257080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed79a8-d8e9-447a-95dc-da3a00a061e2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:13.257372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed79a8-d8e9-447a-95dc-da3a00a061e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:13.257652 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed79a8-d8e9-447a-95dc-da3a00a061e2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:13.258460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed79a8-d8e9-447a-95dc-da3a00a061e2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:13.259855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed79a8-d8e9-447a-95dc-da3a00a061e2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:13.262993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd77c6-588d-4c7d-83fc-ed8cbc3f6573 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:13.263285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd77c6-588d-4c7d-83fc-ed8cbc3f6573 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:13.263566 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd77c6-588d-4c7d-83fc-ed8cbc3f6573 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:13.265225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 458/915] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:13.270218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330577c-44bd-4368-927f-404c5b5e4131 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:13.270503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330577c-44bd-4368-927f-404c5b5e4131 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:13.270783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330577c-44bd-4368-927f-404c5b5e4131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b9ed79a8-d8e9-447a-95dc-da3a00a061e2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:13.272624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:13.273667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 16:31:13.274712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:13.276526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 458/916] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:31:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:13.280563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed79a8-d8e9-447a-95dc-da3a00a061e2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:13.294697 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a330577c-44bd-4368-927f-404c5b5e4131 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:13.296441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a330577c-44bd-4368-927f-404c5b5e4131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:13.314914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:13.361344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:13.391450 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 459/917] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:13.395416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed79a8-d8e9-447a-95dc-da3a00a061e2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:13.414284 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330577c-44bd-4368-927f-404c5b5e4131 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:13.434742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15812c51-702a-46d7-88ec-46199a4d2bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:13.435477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15812c51-702a-46d7-88ec-46199a4d2bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:13.435779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15812c51-702a-46d7-88ec-46199a4d2bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:13.436052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15812c51-702a-46d7-88ec-46199a4d2bd3 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:13.436775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15812c51-702a-46d7-88ec-46199a4d2bd3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:13.437122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15812c51-702a-46d7-88ec-46199a4d2bd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:13.438616 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15812c51-702a-46d7-88ec-46199a4d2bd3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:13.444376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 459/918] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:31:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:13.451014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15812c51-702a-46d7-88ec-46199a4d2bd3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:13.541829 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330577c-44bd-4368-927f-404c5b5e4131 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:13.558054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:13.566042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15812c51-702a-46d7-88ec-46199a4d2bd3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:13.584756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba58547-83a8-4f62-bd86-789d564ccc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:13.585944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba58547-83a8-4f62-bd86-789d564ccc9e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:13.586481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba58547-83a8-4f62-bd86-789d564ccc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:13.586963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba58547-83a8-4f62-bd86-789d564ccc9e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:13.587825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba58547-83a8-4f62-bd86-789d564ccc9e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:13.589511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba58547-83a8-4f62-bd86-789d564ccc9e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-7ba58547-83a8-4f62-bd86-789d564ccc9e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:13.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:13.605033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:13.606410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:13.614053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302eb59-93c6-4ca8-9dc8-a601ca9fe14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:13.614053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302eb59-93c6-4ca8-9dc8-a601ca9fe14a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:13.614053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302eb59-93c6-4ca8-9dc8-a601ca9fe14a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:13.614053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302eb59-93c6-4ca8-9dc8-a601ca9fe14a None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:13.614419 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302eb59-93c6-4ca8-9dc8-a601ca9fe14a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:13.614419 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302eb59-93c6-4ca8-9dc8-a601ca9fe14a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:13.614419 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302eb59-93c6-4ca8-9dc8-a601ca9fe14a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:13.614849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 460/919] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Jul 18 16:31:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:13.616012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba58547-83a8-4f62-bd86-789d564ccc9e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:13.619663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 460/920] 10.222.0.88 () {56 vars in 1114 bytes} [Mon Jul 18 16:31:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:13.625430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302eb59-93c6-4ca8-9dc8-a601ca9fe14a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:13.639198 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:13.732138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba58547-83a8-4f62-bd86-789d564ccc9e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:13.761927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302eb59-93c6-4ca8-9dc8-a601ca9fe14a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:13.773743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220949a8-f9bd-4fa5-a53e-4dd9d7bc4e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:13.774858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220949a8-f9bd-4fa5-a53e-4dd9d7bc4e4d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:13.775590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220949a8-f9bd-4fa5-a53e-4dd9d7bc4e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:13.776539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220949a8-f9bd-4fa5-a53e-4dd9d7bc4e4d None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:13.778344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220949a8-f9bd-4fa5-a53e-4dd9d7bc4e4d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:13.779377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220949a8-f9bd-4fa5-a53e-4dd9d7bc4e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:13.782625 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220949a8-f9bd-4fa5-a53e-4dd9d7bc4e4d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:13.797303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 461/921] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Jul 18 16:31:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:13.799337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220949a8-f9bd-4fa5-a53e-4dd9d7bc4e4d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:13.805010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94df02-3898-49ee-a379-428beb54c957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:13.805685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94df02-3898-49ee-a379-428beb54c957 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:13.805979 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94df02-3898-49ee-a379-428beb54c957 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:13.806262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94df02-3898-49ee-a379-428beb54c957 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:13.807014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94df02-3898-49ee-a379-428beb54c957 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:13.808386 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94df02-3898-49ee-a379-428beb54c957 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:13.895214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:13.897707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 461/922] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Jul 18 16:31:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:31:13.922703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220949a8-f9bd-4fa5-a53e-4dd9d7bc4e4d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:13.969232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3ffc3-5449-474e-ad0b-6cad25e212f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:13.970036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3ffc3-5449-474e-ad0b-6cad25e212f8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:13.970341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3ffc3-5449-474e-ad0b-6cad25e212f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:13.970601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3ffc3-5449-474e-ad0b-6cad25e212f8 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:13.971335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3ffc3-5449-474e-ad0b-6cad25e212f8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:13.971647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3ffc3-5449-474e-ad0b-6cad25e212f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:13.973181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3ffc3-5449-474e-ad0b-6cad25e212f8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:13.979566 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 462/923] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Jul 18 16:31:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:13.986154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94df02-3898-49ee-a379-428beb54c957 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:14.008441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:14.089167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb94f0-e040-49ed-912c-3db587cc25a8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.090268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb94f0-e040-49ed-912c-3db587cc25a8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.090268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb94f0-e040-49ed-912c-3db587cc25a8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.091754 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 463/924] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:14.097585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377d208-29a0-4dde-9151-5a6bcccc3271 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.098416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377d208-29a0-4dde-9151-5a6bcccc3271 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.098937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377d208-29a0-4dde-9151-5a6bcccc3271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.112948 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94df02-3898-49ee-a379-428beb54c957 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:14.124018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e377d208-29a0-4dde-9151-5a6bcccc3271 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:14.126222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e377d208-29a0-4dde-9151-5a6bcccc3271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:14.158627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acec7ab-6ce3-499b-b157-efde50112622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:14.159305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec7ab-6ce3-499b-b157-efde50112622 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.159585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec7ab-6ce3-499b-b157-efde50112622 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.159864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec7ab-6ce3-499b-b157-efde50112622 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.160669 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acec7ab-6ce3-499b-b157-efde50112622 None admin] RBAC: Adding query filter params (interface=public, service_id=8d4d0cb89dcf4dbe96eee61939f6462b, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:14.161685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acec7ab-6ce3-499b-b157-efde50112622 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:14.162405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acec7ab-6ce3-499b-b157-efde50112622 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:14.169382 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 462/925] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Jul 18 16:31:13 2022] GET /identity/v3/endpoints?service_id=8d4d0cb89dcf4dbe96eee61939f6462b&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:14.174795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acec7ab-6ce3-499b-b157-efde50112622 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:14.215375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 464/926] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:14.242815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377d208-29a0-4dde-9151-5a6bcccc3271 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:14.278800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:14.304447 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acec7ab-6ce3-499b-b157-efde50112622 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:14.339452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:14.346409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a54c21-a296-4e62-980c-6f2f5072c602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:14.347420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a54c21-a296-4e62-980c-6f2f5072c602 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.348048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a54c21-a296-4e62-980c-6f2f5072c602 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.353772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a54c21-a296-4e62-980c-6f2f5072c602 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.353772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a54c21-a296-4e62-980c-6f2f5072c602 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:14.353772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a54c21-a296-4e62-980c-6f2f5072c602 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:14.360993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 463/927] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Jul 18 16:31:14 2022] GET /identity/v3/services => generated 1497 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:31:14.380480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377d208-29a0-4dde-9151-5a6bcccc3271 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:14.412243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3985b-9785-42c6-b580-f8fffa0728ad None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.412570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3985b-9785-42c6-b580-f8fffa0728ad None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.412941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3985b-9785-42c6-b580-f8fffa0728ad None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.414615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 464/928] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:14.420483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e97f-3193-4b80-9df3-b3d86bacba3d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.420838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e97f-3193-4b80-9df3-b3d86bacba3d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.421077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e97f-3193-4b80-9df3-b3d86bacba3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.427018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998f25b-af7a-4e5c-b2c7-93f5c9e27a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:14.428182 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998f25b-af7a-4e5c-b2c7-93f5c9e27a51 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.428654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998f25b-af7a-4e5c-b2c7-93f5c9e27a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.429245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998f25b-af7a-4e5c-b2c7-93f5c9e27a51 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.439195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3311e97f-3193-4b80-9df3-b3d86bacba3d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:14.440486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3311e97f-3193-4b80-9df3-b3d86bacba3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:14.443012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998f25b-af7a-4e5c-b2c7-93f5c9e27a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a998f25b-af7a-4e5c-b2c7-93f5c9e27a51 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:14.452411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:14.453487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:14.454328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:14.457086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 465/929] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Jul 18 16:31:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:14.463611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998f25b-af7a-4e5c-b2c7-93f5c9e27a51 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:14.526276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 465/930] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:14.548871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311e97f-3193-4b80-9df3-b3d86bacba3d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:14.588926 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998f25b-af7a-4e5c-b2c7-93f5c9e27a51 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:14.634314 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2855c24-4ddf-43c7-a665-5a889fcae329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:14.635388 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855c24-4ddf-43c7-a665-5a889fcae329 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.635803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855c24-4ddf-43c7-a665-5a889fcae329 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.636232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855c24-4ddf-43c7-a665-5a889fcae329 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.637649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2855c24-4ddf-43c7-a665-5a889fcae329 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:14.637977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2855c24-4ddf-43c7-a665-5a889fcae329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:14.639665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2855c24-4ddf-43c7-a665-5a889fcae329 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:14.645160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:14.653323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:14.660207 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 466/931] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Jul 18 16:31:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:14.665602 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2855c24-4ddf-43c7-a665-5a889fcae329 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:14.687378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311e97f-3193-4b80-9df3-b3d86bacba3d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:14.730596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cc7ae-ba17-49f3-880c-5940887679a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:14.731679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cc7ae-ba17-49f3-880c-5940887679a0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.732094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cc7ae-ba17-49f3-880c-5940887679a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.732484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cc7ae-ba17-49f3-880c-5940887679a0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.733416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cc7ae-ba17-49f3-880c-5940887679a0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:14.735355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cc7ae-ba17-49f3-880c-5940887679a0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d10cc7ae-ba17-49f3-880c-5940887679a0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:14.750807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:14.751812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:14.752646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 466/932] 10.222.0.88 () {56 vars in 1132 bytes} [Mon Jul 18 16:31:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:14.758529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cc7ae-ba17-49f3-880c-5940887679a0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:14.794143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2855c24-4ddf-43c7-a665-5a889fcae329 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:14.832311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fc9cf-1f4f-4012-956a-453b3fff349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:14.833509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fc9cf-1f4f-4012-956a-453b3fff349e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.834029 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fc9cf-1f4f-4012-956a-453b3fff349e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.834518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fc9cf-1f4f-4012-956a-453b3fff349e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.846832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9fc9cf-1f4f-4012-956a-453b3fff349e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:14.848649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9fc9cf-1f4f-4012-956a-453b3fff349e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ee9fc9cf-1f4f-4012-956a-453b3fff349e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:14.867613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:14.868560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:14.869473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:14.869473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:14.869473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:14.869473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:14.876638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cc7ae-ba17-49f3-880c-5940887679a0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:14.876944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 467/933] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:14.878023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fc9cf-1f4f-4012-956a-453b3fff349e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:14.914910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e321a55-8fef-438a-b758-21e99050e19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:14.915499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e321a55-8fef-438a-b758-21e99050e19d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:14.915790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e321a55-8fef-438a-b758-21e99050e19d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:14.916068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e321a55-8fef-438a-b758-21e99050e19d None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:14.916789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e321a55-8fef-438a-b758-21e99050e19d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:14.917275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e321a55-8fef-438a-b758-21e99050e19d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:14.918716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e321a55-8fef-438a-b758-21e99050e19d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:14.920949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:14.927103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 467/934] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Jul 18 16:31:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:14.938070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e321a55-8fef-438a-b758-21e99050e19d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:14.948102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:15.022547 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fc9cf-1f4f-4012-956a-453b3fff349e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:15.052851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e321a55-8fef-438a-b758-21e99050e19d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:15.060176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce12451-a67b-447b-b536-1988c1bfc74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:15.060849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce12451-a67b-447b-b536-1988c1bfc74e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:15.061162 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce12451-a67b-447b-b536-1988c1bfc74e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:15.061435 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce12451-a67b-447b-b536-1988c1bfc74e None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:15.062163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce12451-a67b-447b-b536-1988c1bfc74e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:15.062469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce12451-a67b-447b-b536-1988c1bfc74e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:15.063851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce12451-a67b-447b-b536-1988c1bfc74e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:15.077258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 468/935] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:15.082311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce12451-a67b-447b-b536-1988c1bfc74e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:15.099420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8f89b-b678-4aeb-9187-b238a07d4fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:15.100092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8f89b-b678-4aeb-9187-b238a07d4fff None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:15.100374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8f89b-b678-4aeb-9187-b238a07d4fff None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:15.100648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8f89b-b678-4aeb-9187-b238a07d4fff None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:15.101415 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8f89b-b678-4aeb-9187-b238a07d4fff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:15.101721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8f89b-b678-4aeb-9187-b238a07d4fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:15.103105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8f89b-b678-4aeb-9187-b238a07d4fff None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:15.113792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 468/936] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Jul 18 16:31:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:15.114850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8f89b-b678-4aeb-9187-b238a07d4fff None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:15.204338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce12451-a67b-447b-b536-1988c1bfc74e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:15.213316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:15.213497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:15.225461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8f89b-b678-4aeb-9187-b238a07d4fff None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:15.244278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db986f5-d724-4f6b-baa1-7daa642191d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:15.245559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db986f5-d724-4f6b-baa1-7daa642191d1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:15.246240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db986f5-d724-4f6b-baa1-7daa642191d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:15.246726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db986f5-d724-4f6b-baa1-7daa642191d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:15.255375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db986f5-d724-4f6b-baa1-7daa642191d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:15.257155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db986f5-d724-4f6b-baa1-7daa642191d1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:15.262735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66a14a-dcf1-4374-92ca-a0f2d6907233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:15.263412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66a14a-dcf1-4374-92ca-a0f2d6907233 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:15.263684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66a14a-dcf1-4374-92ca-a0f2d6907233 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:15.263962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66a14a-dcf1-4374-92ca-a0f2d6907233 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:15.264705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66a14a-dcf1-4374-92ca-a0f2d6907233 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:15.268143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66a14a-dcf1-4374-92ca-a0f2d6907233 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2db986f5-d724-4f6b-baa1-7daa642191d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:31:15.276922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:15.278540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:15.280074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:15.280074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:15.280074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:31:15.280074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:31:15.280074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:31:15.280074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:31:15.280074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:31:15.280074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:31:15.280074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:31:15.280074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:15.282140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 469/937] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Jul 18 16:31:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:15.287919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db986f5-d724-4f6b-baa1-7daa642191d1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:15.378768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 469/938] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Jul 18 16:31:15 2022] POST /identity/v3/endpoints => generated 347 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:31:15.401869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db986f5-d724-4f6b-baa1-7daa642191d1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:15.447567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d50f64-a03e-4c05-b80e-e5fb5e4d50df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:15.448254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d50f64-a03e-4c05-b80e-e5fb5e4d50df None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:15.448537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d50f64-a03e-4c05-b80e-e5fb5e4d50df None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:15.448813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d50f64-a03e-4c05-b80e-e5fb5e4d50df None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:15.450322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d50f64-a03e-4c05-b80e-e5fb5e4d50df None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:15.450631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d50f64-a03e-4c05-b80e-e5fb5e4d50df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:15.452120 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d50f64-a03e-4c05-b80e-e5fb5e4d50df None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:15.477312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 470/939] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Jul 18 16:31:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:15.482811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66a14a-dcf1-4374-92ca-a0f2d6907233 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:15.602918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66a14a-dcf1-4374-92ca-a0f2d6907233 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:15.627165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:15.643054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b3729-6f63-418b-b0b1-c8079252f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:15.643875 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b3729-6f63-418b-b0b1-c8079252f620 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:15.645863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b3729-6f63-418b-b0b1-c8079252f620 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:15.645863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b3729-6f63-418b-b0b1-c8079252f620 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:15.655994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b3729-6f63-418b-b0b1-c8079252f620 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:15.657567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b3729-6f63-418b-b0b1-c8079252f620 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d39b3729-6f63-418b-b0b1-c8079252f620 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:15.676274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:15.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:15.682247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:15.682247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:15.682247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:15.682247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:15.682247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 470/940] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:15.684193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d50f64-a03e-4c05-b80e-e5fb5e4d50df None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:15.709655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:15.794441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d50f64-a03e-4c05-b80e-e5fb5e4d50df None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:15.834873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d133d-715c-4f8f-8edc-047a8a4a1e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:15.835569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d133d-715c-4f8f-8edc-047a8a4a1e0a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:15.835939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d133d-715c-4f8f-8edc-047a8a4a1e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:15.836222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d133d-715c-4f8f-8edc-047a8a4a1e0a None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:15.837030 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d133d-715c-4f8f-8edc-047a8a4a1e0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:15.837332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d133d-715c-4f8f-8edc-047a8a4a1e0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:15.838699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d133d-715c-4f8f-8edc-047a8a4a1e0a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:15.851456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 471/941] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:15.857452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b3729-6f63-418b-b0b1-c8079252f620 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:15.968837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b3729-6f63-418b-b0b1-c8079252f620 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:16.006287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b340d0-4725-4600-9414-981c3bca8695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:16.007094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b340d0-4725-4600-9414-981c3bca8695 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:16.007376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b340d0-4725-4600-9414-981c3bca8695 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:16.007654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b340d0-4725-4600-9414-981c3bca8695 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:16.015119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b340d0-4725-4600-9414-981c3bca8695 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:16.016594 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b340d0-4725-4600-9414-981c3bca8695 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-27b340d0-4725-4600-9414-981c3bca8695 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:31:16.031586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:16.032637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:16.033543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:31:16.033543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:31:16.033543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:31:16.033543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:31:16.033543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:31:16.033543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:31:16.033543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:16.033543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:16.037200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 471/942] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:16.038922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d133d-715c-4f8f-8edc-047a8a4a1e0a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:16.050534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a014112-ee9f-4340-aa51-e1783362d7e2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:16.050816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a014112-ee9f-4340-aa51-e1783362d7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:16.051018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a014112-ee9f-4340-aa51-e1783362d7e2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:16.052550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 472/943] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:16.058541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf7807-5373-427c-b8b8-5ad823c68a63 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:16.059149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf7807-5373-427c-b8b8-5ad823c68a63 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:16.059149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf7807-5373-427c-b8b8-5ad823c68a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:16.079326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a0cf7807-5373-427c-b8b8-5ad823c68a63 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:16.080601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a0cf7807-5373-427c-b8b8-5ad823c68a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:16.129619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:16.138527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:16.160909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 473/944] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:16.165155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d133d-715c-4f8f-8edc-047a8a4a1e0a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:16.181560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf7807-5373-427c-b8b8-5ad823c68a63 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:16.204283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259647d-4532-4a09-bafe-3e7fa5c87452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:16.205389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259647d-4532-4a09-bafe-3e7fa5c87452 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:16.206320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259647d-4532-4a09-bafe-3e7fa5c87452 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:16.206824 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259647d-4532-4a09-bafe-3e7fa5c87452 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:16.207838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259647d-4532-4a09-bafe-3e7fa5c87452 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:16.208363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259647d-4532-4a09-bafe-3e7fa5c87452 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:16.210065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259647d-4532-4a09-bafe-3e7fa5c87452 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:16.223558 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 472/945] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:31:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:16.230773 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259647d-4532-4a09-bafe-3e7fa5c87452 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:16.311766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf7807-5373-427c-b8b8-5ad823c68a63 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:16.340956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259647d-4532-4a09-bafe-3e7fa5c87452 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:16.354378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0b304-cb7d-45cd-bf11-9657a0ddf3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:16.356037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b304-cb7d-45cd-bf11-9657a0ddf3b6 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:16.356649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b304-cb7d-45cd-bf11-9657a0ddf3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:16.357237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b304-cb7d-45cd-bf11-9657a0ddf3b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:16.358218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0b304-cb7d-45cd-bf11-9657a0ddf3b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:16.359994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0b304-cb7d-45cd-bf11-9657a0ddf3b6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:16.361733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: WARNING keystone.server.flask.common [None req-10f0b304-cb7d-45cd-bf11-9657a0ddf3b6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 16:31:16.378800 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47180a-4415-48a8-855d-9f7ec8500510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:16.379617 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47180a-4415-48a8-855d-9f7ec8500510 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:16.379926 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47180a-4415-48a8-855d-9f7ec8500510 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:16.380164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47180a-4415-48a8-855d-9f7ec8500510 None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/3f38abef63e24567a995b30c144177fa/roles/443cff586282466c87ffce62f1440b63` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:16.382532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47180a-4415-48a8-855d-9f7ec8500510 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=3f38abef63e24567a995b30c144177fa, role_id=443cff586282466c87ffce62f1440b63)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:16.383971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47180a-4415-48a8-855d-9f7ec8500510 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:16.423731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.sql.core [None req-10f0b304-cb7d-45cd-bf11-9657a0ddf3b6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 16:31:16.423731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 16:31:16.423731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [parameters: {'id': '83ae253ea23743d9a3038aee94c013dd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 16:31:16.423731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80522) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-10f0b304-cb7d-45cd-bf11-9657a0ddf3b6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 83ae253ea23743d9a3038aee94c013dd.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 83ae253ea23743d9a3038aee94c013dd. Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:31:16.431736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result.read() Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 16:31:16.434962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application yield resource Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise self.value Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application session.commit() Jul 18 16:31:16.435925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.session.flush() Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application compat.raise_( Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 16:31:16.436850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 16:31:16.439232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result.read() Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:31:16.441247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [parameters: {'id': '83ae253ea23743d9a3038aee94c013dd', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:16.444665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:16.446725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:16.448849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:16.448849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 16:31:16.448849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 16:31:16.448849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:16.448849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:16.448849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 16:31:16.448849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 16:31:16.448849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 83ae253ea23743d9a3038aee94c013dd. Jul 18 16:31:16.448849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:16.448849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 474/946] 10.222.0.88 () {60 vars in 1140 bytes} [Mon Jul 18 16:31:16 2022] POST /identity/v3/projects => generated 262 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:16.448849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0b304-cb7d-45cd-bf11-9657a0ddf3b6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:16.461149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:16.484659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:16.491080 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 473/947] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:31:16 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/3f38abef63e24567a995b30c144177fa/roles/443cff586282466c87ffce62f1440b63 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:31:16.525717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff281825-b711-4b9e-a064-4e2236d00147 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:16.526125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff281825-b711-4b9e-a064-4e2236d00147 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:16.526391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff281825-b711-4b9e-a064-4e2236d00147 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:16.532679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 474/948] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:16.534274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81958bd3-6847-46d0-b910-ccb3ccd4ed08 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:16.534572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81958bd3-6847-46d0-b910-ccb3ccd4ed08 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:16.534852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81958bd3-6847-46d0-b910-ccb3ccd4ed08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:16.553639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-81958bd3-6847-46d0-b910-ccb3ccd4ed08 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:16.554942 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-81958bd3-6847-46d0-b910-ccb3ccd4ed08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:16.574221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0b304-cb7d-45cd-bf11-9657a0ddf3b6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:16.630321 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e0db8-d1cc-4fee-89ce-96f3ec018a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:16.633178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e0db8-d1cc-4fee-89ce-96f3ec018a69 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:16.634456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e0db8-d1cc-4fee-89ce-96f3ec018a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:16.634933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e0db8-d1cc-4fee-89ce-96f3ec018a69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:16.643531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e0db8-d1cc-4fee-89ce-96f3ec018a69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:16.644163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 475/949] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:16.645702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e0db8-d1cc-4fee-89ce-96f3ec018a69 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-662e0db8-d1cc-4fee-89ce-96f3ec018a69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:31:16.661681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:16.662826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:16.663914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:31:16.663914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:31:16.663914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:31:16.663914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:31:16.663914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:31:16.663914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:31:16.663914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:16.663914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:16.665904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 475/950] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:16.667090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81958bd3-6847-46d0-b910-ccb3ccd4ed08 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:16.674160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e0db8-d1cc-4fee-89ce-96f3ec018a69 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:16.785353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:16.805046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81958bd3-6847-46d0-b910-ccb3ccd4ed08 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:16.812656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e0db8-d1cc-4fee-89ce-96f3ec018a69 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:16.837227 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:16.844102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ac8a9-379b-41f8-aa0c-611be9fcb0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:16.845185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ac8a9-379b-41f8-aa0c-611be9fcb0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:16.845628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ac8a9-379b-41f8-aa0c-611be9fcb0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:16.846232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ac8a9-379b-41f8-aa0c-611be9fcb0d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:16.847058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ac8a9-379b-41f8-aa0c-611be9fcb0d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:16.848612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ac8a9-379b-41f8-aa0c-611be9fcb0d7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:16.850387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd65cb-c91e-4175-a142-9691c10e0897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:16.851168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd65cb-c91e-4175-a142-9691c10e0897 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:16.851493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd65cb-c91e-4175-a142-9691c10e0897 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:16.851842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd65cb-c91e-4175-a142-9691c10e0897 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:16.852773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd65cb-c91e-4175-a142-9691c10e0897 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:16.853144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd65cb-c91e-4175-a142-9691c10e0897 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:16.854683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd65cb-c91e-4175-a142-9691c10e0897 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4f4ac8a9-379b-41f8-aa0c-611be9fcb0d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:16.863975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:16.865392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:16.866510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:16.869462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 476/951] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Jul 18 16:31:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:16.881109 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ac8a9-379b-41f8-aa0c-611be9fcb0d7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:16.888921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 476/952] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:31:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:17.004033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ac8a9-379b-41f8-aa0c-611be9fcb0d7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:17.045836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf13c0-c7f3-44f3-b1ea-ea6849c6ea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:17.046511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf13c0-c7f3-44f3-b1ea-ea6849c6ea27 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:17.046860 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf13c0-c7f3-44f3-b1ea-ea6849c6ea27 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:17.047143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf13c0-c7f3-44f3-b1ea-ea6849c6ea27 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:17.047908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf13c0-c7f3-44f3-b1ea-ea6849c6ea27 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:17.048220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf13c0-c7f3-44f3-b1ea-ea6849c6ea27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:17.049646 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf13c0-c7f3-44f3-b1ea-ea6849c6ea27 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:17.057054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 477/953] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Jul 18 16:31:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:17.061006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd65cb-c91e-4175-a142-9691c10e0897 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:17.182884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd65cb-c91e-4175-a142-9691c10e0897 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:17.221811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d2ac3-7337-4d2d-bbc7-537b430701ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:17.222490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d2ac3-7337-4d2d-bbc7-537b430701ec None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:17.222777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d2ac3-7337-4d2d-bbc7-537b430701ec None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:17.223058 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d2ac3-7337-4d2d-bbc7-537b430701ec None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:17.223777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d2ac3-7337-4d2d-bbc7-537b430701ec None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:17.224087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d2ac3-7337-4d2d-bbc7-537b430701ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:17.225506 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d2ac3-7337-4d2d-bbc7-537b430701ec None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:17.231711 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 477/954] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Jul 18 16:31:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:17.237411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf13c0-c7f3-44f3-b1ea-ea6849c6ea27 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:17.303402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:17.348686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf13c0-c7f3-44f3-b1ea-ea6849c6ea27 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:17.384194 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e247b-a98f-43fb-bd96-f12d30a6b5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:17.384913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e247b-a98f-43fb-bd96-f12d30a6b5c9 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:17.385276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e247b-a98f-43fb-bd96-f12d30a6b5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:17.385556 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e247b-a98f-43fb-bd96-f12d30a6b5c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:17.386326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e247b-a98f-43fb-bd96-f12d30a6b5c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:17.388256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e247b-a98f-43fb-bd96-f12d30a6b5c9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:17.461591 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 478/955] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Jul 18 16:31:17 2022] POST /identity/v3/endpoints => generated 349 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:31:17.608353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc9a99-179b-43cd-b373-75738424f806 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:17.608700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc9a99-179b-43cd-b373-75738424f806 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:17.608927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc9a99-179b-43cd-b373-75738424f806 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:17.610327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 478/956] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:17.615963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672045bc-dce7-4374-acbd-efa349ced9d9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:17.616256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672045bc-dce7-4374-acbd-efa349ced9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:17.616539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672045bc-dce7-4374-acbd-efa349ced9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:17.634888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-672045bc-dce7-4374-acbd-efa349ced9d9 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:17.636082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-672045bc-dce7-4374-acbd-efa349ced9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:17.735289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 479/957] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:17.754224 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc9a99-179b-43cd-b373-75738424f806 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:17.757240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:17.869988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc9a99-179b-43cd-b373-75738424f806 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:17.907027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1619f-830a-4e7c-badc-2e7efae2b156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:17.908522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1619f-830a-4e7c-badc-2e7efae2b156 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:17.908999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1619f-830a-4e7c-badc-2e7efae2b156 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:17.909443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1619f-830a-4e7c-badc-2e7efae2b156 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:17.910243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1619f-830a-4e7c-badc-2e7efae2b156 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:17.911821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1619f-830a-4e7c-badc-2e7efae2b156 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-3ce1619f-830a-4e7c-badc-2e7efae2b156 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:17.924330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:17.925598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:31:17.926541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:17.928941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 479/958] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:17.943061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672045bc-dce7-4374-acbd-efa349ced9d9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:17.987985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:18.053338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672045bc-dce7-4374-acbd-efa349ced9d9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:18.090434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995d81b-28bf-4683-8110-fd6f7ec6b61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:18.091110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d81b-28bf-4683-8110-fd6f7ec6b61c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:18.091394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d81b-28bf-4683-8110-fd6f7ec6b61c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:18.091671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995d81b-28bf-4683-8110-fd6f7ec6b61c None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:18.092387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995d81b-28bf-4683-8110-fd6f7ec6b61c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:18.092693 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995d81b-28bf-4683-8110-fd6f7ec6b61c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:18.094099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995d81b-28bf-4683-8110-fd6f7ec6b61c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:18.099399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 480/959] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:31:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:18.105061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1619f-830a-4e7c-badc-2e7efae2b156 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:18.184251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:18.228013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1619f-830a-4e7c-badc-2e7efae2b156 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:18.266611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44491d20-8f9e-43a0-803c-3f5c21970e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:18.267688 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44491d20-8f9e-43a0-803c-3f5c21970e8c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:18.268180 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44491d20-8f9e-43a0-803c-3f5c21970e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:18.268657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44491d20-8f9e-43a0-803c-3f5c21970e8c None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:18.269655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44491d20-8f9e-43a0-803c-3f5c21970e8c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:18.270185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44491d20-8f9e-43a0-803c-3f5c21970e8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:18.271782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44491d20-8f9e-43a0-803c-3f5c21970e8c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:18.281003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 480/960] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:31:18 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:18.642873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104a250-f7e3-4156-936c-44c70fffa4ed None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:18.643374 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104a250-f7e3-4156-936c-44c70fffa4ed None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:18.643374 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104a250-f7e3-4156-936c-44c70fffa4ed None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:18.645088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 481/961] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:18.651113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdace9-054e-432f-82b4-ae3e9c55be82 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:18.651420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdace9-054e-432f-82b4-ae3e9c55be82 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:18.651693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdace9-054e-432f-82b4-ae3e9c55be82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:18.670607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d8bdace9-054e-432f-82b4-ae3e9c55be82 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:18.672146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d8bdace9-054e-432f-82b4-ae3e9c55be82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:18.745365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 481/962] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:18.768981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104a250-f7e3-4156-936c-44c70fffa4ed None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:18.882392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104a250-f7e3-4156-936c-44c70fffa4ed None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:18.885224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:18.921126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b02144-4c7d-4015-9a94-bd21ddcd351f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:18.922287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b02144-4c7d-4015-9a94-bd21ddcd351f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:18.922783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b02144-4c7d-4015-9a94-bd21ddcd351f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:18.923258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b02144-4c7d-4015-9a94-bd21ddcd351f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:18.946542 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b02144-4c7d-4015-9a94-bd21ddcd351f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d3b02144-4c7d-4015-9a94-bd21ddcd351f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:18.958379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:18.959298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:18.960137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:18.962917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 482/963] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Jul 18 16:31:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:18.969300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdace9-054e-432f-82b4-ae3e9c55be82 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:19.035362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:19.079604 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdace9-054e-432f-82b4-ae3e9c55be82 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:19.118337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa623011-c883-4d2a-a0ad-2313a011d7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:19.119861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa623011-c883-4d2a-a0ad-2313a011d7ad None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:19.120376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa623011-c883-4d2a-a0ad-2313a011d7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:19.121418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa623011-c883-4d2a-a0ad-2313a011d7ad None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:19.122447 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa623011-c883-4d2a-a0ad-2313a011d7ad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:19.122969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa623011-c883-4d2a-a0ad-2313a011d7ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:19.124544 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa623011-c883-4d2a-a0ad-2313a011d7ad None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:19.134273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 482/964] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Jul 18 16:31:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:19.141961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b02144-4c7d-4015-9a94-bd21ddcd351f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:19.144263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481decb-9f45-4e14-8b64-e75ea88bcc90 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:19.144554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481decb-9f45-4e14-8b64-e75ea88bcc90 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:19.144832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481decb-9f45-4e14-8b64-e75ea88bcc90 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:19.146489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 483/965] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:19.152390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a440f-b271-4eed-aca4-a82083a25ee5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:19.152678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a440f-b271-4eed-aca4-a82083a25ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:19.152983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a440f-b271-4eed-aca4-a82083a25ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:19.170364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b46a440f-b271-4eed-aca4-a82083a25ee5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:19.172076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b46a440f-b271-4eed-aca4-a82083a25ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:19.256910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 484/966] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:19.270011 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b02144-4c7d-4015-9a94-bd21ddcd351f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:19.280728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a440f-b271-4eed-aca4-a82083a25ee5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:19.309394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8da482-ca5b-428e-9c08-78d9f3c9fb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:19.310511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8da482-ca5b-428e-9c08-78d9f3c9fb3d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:19.311064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8da482-ca5b-428e-9c08-78d9f3c9fb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:19.311344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8da482-ca5b-428e-9c08-78d9f3c9fb3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:19.325349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8da482-ca5b-428e-9c08-78d9f3c9fb3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:19.327226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8da482-ca5b-428e-9c08-78d9f3c9fb3d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:19.339725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4c8da482-ca5b-428e-9c08-78d9f3c9fb3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:19.349155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:19.353479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:19.354572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:19.354572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:19.354572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:19.354572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:19.354572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 483/967] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:19.357004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8da482-ca5b-428e-9c08-78d9f3c9fb3d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:19.379267 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:19.402491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a440f-b271-4eed-aca4-a82083a25ee5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:19.442156 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465853f-1350-4078-b9c4-8593c1ca7145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:19.443221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465853f-1350-4078-b9c4-8593c1ca7145 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:19.443648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465853f-1350-4078-b9c4-8593c1ca7145 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:19.444180 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465853f-1350-4078-b9c4-8593c1ca7145 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:19.445063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465853f-1350-4078-b9c4-8593c1ca7145 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:19.446689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465853f-1350-4078-b9c4-8593c1ca7145 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:19.477546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8da482-ca5b-428e-9c08-78d9f3c9fb3d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:19.493923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.sql.core [None req-2465853f-1350-4078-b9c4-8593c1ca7145 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 16:31:19.493923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 16:31:19.493923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [parameters: {'user_id': 'a25c3cb00d554c4bb8296a9b4bdf9927', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 16:31:19.493923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80522) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2465853f-1350-4078-b9c4-8593c1ca7145 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result.read() Jul 18 16:31:19.506605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:31:19.507696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application yield resource Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application next(self.gen) Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise self.value Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application session.commit() Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 16:31:19.508723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.session.flush() Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application compat.raise_( Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 16:31:19.509713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 16:31:19.510641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application conn.query(q) Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application result.read() Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 16:31:19.512229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a25c3cb00d554c4bb8296a9b4bdf9927', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:19.516452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 16:31:19.517493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 16:31:19.518491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 16:31:19.518491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 16:31:19.518491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 16:31:19.518491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 16:31:19.518491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 16:31:19.518491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:19.518491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 485/968] 10.222.0.88 () {60 vars in 1131 bytes} [Mon Jul 18 16:31:19 2022] POST /identity/v3/users => generated 160 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:19.521627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465853f-1350-4078-b9c4-8593c1ca7145 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:19.534068 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92544fff-e370-4f28-8d39-b876ce952aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:19.534890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544fff-e370-4f28-8d39-b876ce952aff None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:19.535737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544fff-e370-4f28-8d39-b876ce952aff None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:19.536122 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92544fff-e370-4f28-8d39-b876ce952aff None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:19.537012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92544fff-e370-4f28-8d39-b876ce952aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:19.537845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92544fff-e370-4f28-8d39-b876ce952aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:19.539682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92544fff-e370-4f28-8d39-b876ce952aff None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:19.562938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 484/969] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:19.565907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92544fff-e370-4f28-8d39-b876ce952aff None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:19.650973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465853f-1350-4078-b9c4-8593c1ca7145 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:19.653737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:19.673001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:19.687271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92544fff-e370-4f28-8d39-b876ce952aff None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:19.696728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f2352-b441-4b52-894f-bfd3c975562d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:19.698426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f2352-b441-4b52-894f-bfd3c975562d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:19.707295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f2352-b441-4b52-894f-bfd3c975562d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:19.707295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f2352-b441-4b52-894f-bfd3c975562d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:19.708541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f2352-b441-4b52-894f-bfd3c975562d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:19.710281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f2352-b441-4b52-894f-bfd3c975562d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:19.725662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c3604a-0dc7-467d-9ee7-073f29461f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:19.726514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3604a-0dc7-467d-9ee7-073f29461f77 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:19.726872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3604a-0dc7-467d-9ee7-073f29461f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:19.727228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c3604a-0dc7-467d-9ee7-073f29461f77 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-5d4f2352-b441-4b52-894f-bfd3c975562d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:31:19.727379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:19.728286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:19.729302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:19.729302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:19.729302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:31:19.729302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:31:19.729302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:31:19.729302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:31:19.729302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:31:19.729302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:31:19.729302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 16:31:19.729302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:19.735461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 486/970] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:31:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:19.736822 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e18c1-d264-4f36-8d05-166e5aaa2e08 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:19.737175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e18c1-d264-4f36-8d05-166e5aaa2e08 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:19.737522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e18c1-d264-4f36-8d05-166e5aaa2e08 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:19.739763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3604a-0dc7-467d-9ee7-073f29461f77 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:19.739763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c3604a-0dc7-467d-9ee7-073f29461f77 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:19.743490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 487/971] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:19.743490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e18c1-d264-4f36-8d05-166e5aaa2e08 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-f3c3604a-0dc7-467d-9ee7-073f29461f77 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:31:19.764398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:19.772073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 485/972] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Jul 18 16:31:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5544cc-6a81-42cd-93f0-c78c7e8b0a49 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:19.774344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5544cc-6a81-42cd-93f0-c78c7e8b0a49 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:19.777079 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5544cc-6a81-42cd-93f0-c78c7e8b0a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:19.785671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ad5544cc-6a81-42cd-93f0-c78c7e8b0a49 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:19.786886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ad5544cc-6a81-42cd-93f0-c78c7e8b0a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:19.864774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e18c1-d264-4f36-8d05-166e5aaa2e08 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:19.868928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 486/973] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:19.869950 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5544cc-6a81-42cd-93f0-c78c7e8b0a49 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:19.909925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017f4b1-47eb-4b4e-bd64-7ca65d22957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:19.910618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017f4b1-47eb-4b4e-bd64-7ca65d22957f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:19.910901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017f4b1-47eb-4b4e-bd64-7ca65d22957f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:19.911175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017f4b1-47eb-4b4e-bd64-7ca65d22957f None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:19.911989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017f4b1-47eb-4b4e-bd64-7ca65d22957f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:19.912296 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017f4b1-47eb-4b4e-bd64-7ca65d22957f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:19.913772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017f4b1-47eb-4b4e-bd64-7ca65d22957f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:19.952675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 488/974] 10.222.0.88 () {56 vars in 1101 bytes} [Mon Jul 18 16:31:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:19.953313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017f4b1-47eb-4b4e-bd64-7ca65d22957f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:19.990026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:20.001009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:20.040022 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5544cc-6a81-42cd-93f0-c78c7e8b0a49 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:20.092481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d9ecf-1c31-49c8-a23e-1b9e2e40392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:20.093721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d9ecf-1c31-49c8-a23e-1b9e2e40392e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.094165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d9ecf-1c31-49c8-a23e-1b9e2e40392e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.094584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d9ecf-1c31-49c8-a23e-1b9e2e40392e None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.094964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017f4b1-47eb-4b4e-bd64-7ca65d22957f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:20.095806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d9ecf-1c31-49c8-a23e-1b9e2e40392e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:20.096271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d9ecf-1c31-49c8-a23e-1b9e2e40392e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:20.098144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d9ecf-1c31-49c8-a23e-1b9e2e40392e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:20.130103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 487/975] 10.222.0.88 () {56 vars in 1149 bytes} [Mon Jul 18 16:31:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:20.130678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d9ecf-1c31-49c8-a23e-1b9e2e40392e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:20.154356 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1ddfb-13bb-46f0-a286-c276ef434709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:20.155810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1ddfb-13bb-46f0-a286-c276ef434709 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.156122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1ddfb-13bb-46f0-a286-c276ef434709 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.156378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1ddfb-13bb-46f0-a286-c276ef434709 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.158914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1ddfb-13bb-46f0-a286-c276ef434709 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:20.158914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1ddfb-13bb-46f0-a286-c276ef434709 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2be1ddfb-13bb-46f0-a286-c276ef434709 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:20.175412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:20.176588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:20.177479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 489/976] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Jul 18 16:31:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:20.183267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1ddfb-13bb-46f0-a286-c276ef434709 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:20.276468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d9ecf-1c31-49c8-a23e-1b9e2e40392e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:20.306065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:20.323117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65ff2e-3ca9-4ce9-b04e-5f51b68a9a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:20.324802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65ff2e-3ca9-4ce9-b04e-5f51b68a9a74 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.325346 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65ff2e-3ca9-4ce9-b04e-5f51b68a9a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.326452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65ff2e-3ca9-4ce9-b04e-5f51b68a9a74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.348316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:20.355465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1ddfb-13bb-46f0-a286-c276ef434709 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:20.397384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34b633-b86a-42eb-8324-b89838a5f75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:20.398056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34b633-b86a-42eb-8324-b89838a5f75e None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.398343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34b633-b86a-42eb-8324-b89838a5f75e None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.398618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34b633-b86a-42eb-8324-b89838a5f75e None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.399336 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34b633-b86a-42eb-8324-b89838a5f75e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:20.399643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34b633-b86a-42eb-8324-b89838a5f75e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:20.401096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34b633-b86a-42eb-8324-b89838a5f75e None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:20.407019 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 490/977] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Jul 18 16:31:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:20.412808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34b633-b86a-42eb-8324-b89838a5f75e None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:20.427211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65ff2e-3ca9-4ce9-b04e-5f51b68a9a74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:20.429615 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65ff2e-3ca9-4ce9-b04e-5f51b68a9a74 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:20.541728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34b633-b86a-42eb-8324-b89838a5f75e None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:20.543289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 488/978] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:20 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:31:20.549139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65ff2e-3ca9-4ce9-b04e-5f51b68a9a74 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:20.576863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:20.588409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112103c-212b-4e0c-aac6-d99f2760c388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:20.589216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112103c-212b-4e0c-aac6-d99f2760c388 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.589525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112103c-212b-4e0c-aac6-d99f2760c388 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.590385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112103c-212b-4e0c-aac6-d99f2760c388 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.591104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112103c-212b-4e0c-aac6-d99f2760c388 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:20.591409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112103c-212b-4e0c-aac6-d99f2760c388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:20.592784 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112103c-212b-4e0c-aac6-d99f2760c388 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:20.598413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 491/979] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Jul 18 16:31:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:20.604086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112103c-212b-4e0c-aac6-d99f2760c388 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:20.633145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:20.665982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65ff2e-3ca9-4ce9-b04e-5f51b68a9a74 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:20.704302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea802c-3f64-4204-a780-719d9535b7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:20.705151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea802c-3f64-4204-a780-719d9535b7ea None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.705431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea802c-3f64-4204-a780-719d9535b7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.705707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea802c-3f64-4204-a780-719d9535b7ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.713231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112103c-212b-4e0c-aac6-d99f2760c388 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:20.718103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea802c-3f64-4204-a780-719d9535b7ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:20.719613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea802c-3f64-4204-a780-719d9535b7ea None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2dea802c-3f64-4204-a780-719d9535b7ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:20.740257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:20.741247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:20.742409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:20.742409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:20.742409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:20.742409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:20.742409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 489/980] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:20.743222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa58868-92be-41fe-91f2-68bbf5823a51 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.743442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa58868-92be-41fe-91f2-68bbf5823a51 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.743719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa58868-92be-41fe-91f2-68bbf5823a51 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.745001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 490/981] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:20.748057 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa58868-92be-41fe-91f2-68bbf5823a51 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:20.754268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe0839-cd4d-43c0-91a3-5b54fca58cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:20.754933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe0839-cd4d-43c0-91a3-5b54fca58cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.755218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe0839-cd4d-43c0-91a3-5b54fca58cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.755494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe0839-cd4d-43c0-91a3-5b54fca58cc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.756298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe0839-cd4d-43c0-91a3-5b54fca58cc9 None admin] RBAC: Adding query filter params (interface=admin, service_id=8d4d0cb89dcf4dbe96eee61939f6462b, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:20.756617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe0839-cd4d-43c0-91a3-5b54fca58cc9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:20.758357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe0839-cd4d-43c0-91a3-5b54fca58cc9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:20.764550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 492/982] 10.222.0.88 () {56 vars in 1249 bytes} [Mon Jul 18 16:31:20 2022] GET /identity/v3/endpoints?service_id=8d4d0cb89dcf4dbe96eee61939f6462b&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:20.765629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3404c75-c5c0-4caa-8f57-1b7eecf997bb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.765990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3404c75-c5c0-4caa-8f57-1b7eecf997bb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.766274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3404c75-c5c0-4caa-8f57-1b7eecf997bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.783017 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c3404c75-c5c0-4caa-8f57-1b7eecf997bb None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:20.786153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c3404c75-c5c0-4caa-8f57-1b7eecf997bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:20.829377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:20.851808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa58868-92be-41fe-91f2-68bbf5823a51 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:20.853177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 493/983] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:20.853835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3404c75-c5c0-4caa-8f57-1b7eecf997bb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:20.873852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:20.888247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8be1e-4c07-4a5e-a2f9-7e5d607e5c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:20.888979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8be1e-4c07-4a5e-a2f9-7e5d607e5c72 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.889264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8be1e-4c07-4a5e-a2f9-7e5d607e5c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.889990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8be1e-4c07-4a5e-a2f9-7e5d607e5c72 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.890250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8be1e-4c07-4a5e-a2f9-7e5d607e5c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:20.890554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8be1e-4c07-4a5e-a2f9-7e5d607e5c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:20.891975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8be1e-4c07-4a5e-a2f9-7e5d607e5c72 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:20.904298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 491/984] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:20.905135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8be1e-4c07-4a5e-a2f9-7e5d607e5c72 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:20.956497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3404c75-c5c0-4caa-8f57-1b7eecf997bb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:20.991065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423a325-78c5-496f-b19f-caee4294c1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:20.992241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423a325-78c5-496f-b19f-caee4294c1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:20.992526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423a325-78c5-496f-b19f-caee4294c1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:20.992802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423a325-78c5-496f-b19f-caee4294c1e1 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:20.993520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423a325-78c5-496f-b19f-caee4294c1e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:20.994893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423a325-78c5-496f-b19f-caee4294c1e1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:21.000974 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 494/985] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Jul 18 16:31:20 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:31:21.001774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423a325-78c5-496f-b19f-caee4294c1e1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:21.016934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8be1e-4c07-4a5e-a2f9-7e5d607e5c72 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:21.058924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6c659-853d-4717-85a4-26b077e4c755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:21.059618 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6c659-853d-4717-85a4-26b077e4c755 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:21.059901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6c659-853d-4717-85a4-26b077e4c755 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:21.060178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6c659-853d-4717-85a4-26b077e4c755 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:21.060956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6c659-853d-4717-85a4-26b077e4c755 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:21.062372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6c659-853d-4717-85a4-26b077e4c755 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:21.066148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:21.102118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:21.120430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423a325-78c5-496f-b19f-caee4294c1e1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:21.157485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d9177-343f-4a00-854d-cae82b26aa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:21.158284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9177-343f-4a00-854d-cae82b26aa12 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:21.158568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9177-343f-4a00-854d-cae82b26aa12 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:21.158843 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d9177-343f-4a00-854d-cae82b26aa12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:21.166485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d9177-343f-4a00-854d-cae82b26aa12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:21.167963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d9177-343f-4a00-854d-cae82b26aa12 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ba9d9177-343f-4a00-854d-cae82b26aa12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:31:21.183854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:21.185560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:21.186673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:31:21.186673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:31:21.186673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:31:21.186673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:31:21.186673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:31:21.186673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:31:21.186673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:21.186673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:21.186673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 495/986] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:21.190277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 492/987] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:31:20 2022] POST /identity/v3/services => generated 242 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:21.190837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d9177-343f-4a00-854d-cae82b26aa12 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:21.312115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d9177-343f-4a00-854d-cae82b26aa12 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:21.361051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2a844-0da6-48b2-a09a-3a982153da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:21.361051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a844-0da6-48b2-a09a-3a982153da49 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:21.361051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a844-0da6-48b2-a09a-3a982153da49 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:21.361051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a844-0da6-48b2-a09a-3a982153da49 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:21.362059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2a844-0da6-48b2-a09a-3a982153da49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:21.362485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2a844-0da6-48b2-a09a-3a982153da49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:21.364026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2a844-0da6-48b2-a09a-3a982153da49 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:21.373459 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:21.379073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 496/988] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:31:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:21.385082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6c659-853d-4717-85a4-26b077e4c755 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:21.496684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6c659-853d-4717-85a4-26b077e4c755 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:21.530846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4105a-59f3-41b4-8ce9-a1c34a14ec02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:21.531503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4105a-59f3-41b4-8ce9-a1c34a14ec02 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:21.531782 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4105a-59f3-41b4-8ce9-a1c34a14ec02 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:21.532055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4105a-59f3-41b4-8ce9-a1c34a14ec02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:21.532915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4105a-59f3-41b4-8ce9-a1c34a14ec02 None admin] RBAC: Adding query filter params (role.id=443cff586282466c87ffce62f1440b63, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=3f38abef63e24567a995b30c144177fa) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:21.533213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4105a-59f3-41b4-8ce9-a1c34a14ec02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:21.534644 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4105a-59f3-41b4-8ce9-a1c34a14ec02 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:21.540909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 493/989] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:21 2022] GET /identity/v3/role_assignments?user.id=3f38abef63e24567a995b30c144177fa&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=443cff586282466c87ffce62f1440b63 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:23.031846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a74d8-de5a-459a-bc79-33a2ce3a0694 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.032401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a74d8-de5a-459a-bc79-33a2ce3a0694 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.032401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a74d8-de5a-459a-bc79-33a2ce3a0694 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.033835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 497/990] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:23.039692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f953c4-6e1c-4b21-a6fb-880d43ccecbc None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.040273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f953c4-6e1c-4b21-a6fb-880d43ccecbc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.040273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f953c4-6e1c-4b21-a6fb-880d43ccecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.057530 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-97f953c4-6e1c-4b21-a6fb-880d43ccecbc None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:23.059238 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-97f953c4-6e1c-4b21-a6fb-880d43ccecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:23.118175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:23.130188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 494/991] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:23.153349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a74d8-de5a-459a-bc79-33a2ce3a0694 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:23.264257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a74d8-de5a-459a-bc79-33a2ce3a0694 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:23.266609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aec69-10c0-4b7b-af37-a56c64426069 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.266909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aec69-10c0-4b7b-af37-a56c64426069 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.267189 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1aec69-10c0-4b7b-af37-a56c64426069 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.268569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 495/992] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:23.273614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a8b3a-9f9e-4b62-8820-534fd5c8b3f3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.273904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a8b3a-9f9e-4b62-8820-534fd5c8b3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.274186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a8b3a-9f9e-4b62-8820-534fd5c8b3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.291585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-270a8b3a-9f9e-4b62-8820-534fd5c8b3f3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:23.293370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-270a8b3a-9f9e-4b62-8820-534fd5c8b3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:23.306291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ff6aa-8477-4762-b56f-d9443d278af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:23.307521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ff6aa-8477-4762-b56f-d9443d278af5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.308921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ff6aa-8477-4762-b56f-d9443d278af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.309538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ff6aa-8477-4762-b56f-d9443d278af5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.310478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ff6aa-8477-4762-b56f-d9443d278af5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:23.312659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ff6aa-8477-4762-b56f-d9443d278af5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d09ff6aa-8477-4762-b56f-d9443d278af5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:23.330823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:23.332171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 16:31:23.333246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:23.337067 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 498/993] 10.222.0.88 () {56 vars in 1126 bytes} [Mon Jul 18 16:31:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:23.339351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ff6aa-8477-4762-b56f-d9443d278af5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:23.373386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 496/994] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:23.396135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a8b3a-9f9e-4b62-8820-534fd5c8b3f3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:23.417315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:23.465272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ff6aa-8477-4762-b56f-d9443d278af5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:23.501811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459161d-9cf9-42b0-8183-3fe159e05063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:23.502485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459161d-9cf9-42b0-8183-3fe159e05063 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.502775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459161d-9cf9-42b0-8183-3fe159e05063 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.503056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459161d-9cf9-42b0-8183-3fe159e05063 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.503798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459161d-9cf9-42b0-8183-3fe159e05063 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:23.504111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459161d-9cf9-42b0-8183-3fe159e05063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:23.504207 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a8b3a-9f9e-4b62-8820-534fd5c8b3f3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:23.505585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459161d-9cf9-42b0-8183-3fe159e05063 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:23.510896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 499/995] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Jul 18 16:31:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:23.516925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459161d-9cf9-42b0-8183-3fe159e05063 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:23.529295 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:23.543504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfde6fd-902e-421f-ab5c-931b466186d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:23.544312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfde6fd-902e-421f-ab5c-931b466186d9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.544603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfde6fd-902e-421f-ab5c-931b466186d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.548672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfde6fd-902e-421f-ab5c-931b466186d9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.548672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfde6fd-902e-421f-ab5c-931b466186d9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:23.548672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfde6fd-902e-421f-ab5c-931b466186d9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-2dfde6fd-902e-421f-ab5c-931b466186d9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:23.559825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:23.560922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:31:23.561775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:23.566206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 497/996] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:23.567145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfde6fd-902e-421f-ab5c-931b466186d9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:23.641244 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459161d-9cf9-42b0-8183-3fe159e05063 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:23.668498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:23.675309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfde6fd-902e-421f-ab5c-931b466186d9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:23.677168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43d18f-42c2-4eeb-8327-a5733f1c6221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:23.677845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43d18f-42c2-4eeb-8327-a5733f1c6221 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.678128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43d18f-42c2-4eeb-8327-a5733f1c6221 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.678408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43d18f-42c2-4eeb-8327-a5733f1c6221 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.685976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43d18f-42c2-4eeb-8327-a5733f1c6221 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:23.685976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43d18f-42c2-4eeb-8327-a5733f1c6221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:23.685976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43d18f-42c2-4eeb-8327-a5733f1c6221 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:23.686378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 500/997] 10.222.0.88 () {56 vars in 1122 bytes} [Mon Jul 18 16:31:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:23.687640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8321d76-6628-403a-b0d2-44b9311637a2 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.687930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8321d76-6628-403a-b0d2-44b9311637a2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.688210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8321d76-6628-403a-b0d2-44b9311637a2 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.689511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 501/998] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:23.691912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8321d76-6628-403a-b0d2-44b9311637a2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:23.716611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc3205-4d72-44e5-be15-5f4c1badd844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:23.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc3205-4d72-44e5-be15-5f4c1badd844 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.717703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc3205-4d72-44e5-be15-5f4c1badd844 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.717976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc3205-4d72-44e5-be15-5f4c1badd844 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.718680 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc3205-4d72-44e5-be15-5f4c1badd844 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:23.718985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc3205-4d72-44e5-be15-5f4c1badd844 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:23.720359 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc3205-4d72-44e5-be15-5f4c1badd844 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:23.725341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 498/999] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:31:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:23.726509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4904b3-bc69-4d71-a3f8-a07908e3944f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.726797 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4904b3-bc69-4d71-a3f8-a07908e3944f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.727090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4904b3-bc69-4d71-a3f8-a07908e3944f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.743297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1c4904b3-bc69-4d71-a3f8-a07908e3944f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:23.744489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1c4904b3-bc69-4d71-a3f8-a07908e3944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:23.790171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:23.800120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8321d76-6628-403a-b0d2-44b9311637a2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:23.814349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 499/1000] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:23.815332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4904b3-bc69-4d71-a3f8-a07908e3944f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:23.839940 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d4673-b5fb-4604-9334-39a35f357ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:23.840710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d4673-b5fb-4604-9334-39a35f357ecf None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:23.841030 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d4673-b5fb-4604-9334-39a35f357ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:23.841315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d4673-b5fb-4604-9334-39a35f357ecf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:23.842069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d4673-b5fb-4604-9334-39a35f357ecf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:23.843554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d4673-b5fb-4604-9334-39a35f357ecf None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:23.917966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4904b3-bc69-4d71-a3f8-a07908e3944f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:23.948789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 502/1001] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Jul 18 16:31:23 2022] POST /identity/v3/endpoints => generated 325 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:31:23.948789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d4673-b5fb-4604-9334-39a35f357ecf None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:24.002030 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:24.012073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fecf8-c906-42f3-85d4-6c0aff6ee48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:24.012762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fecf8-c906-42f3-85d4-6c0aff6ee48e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:24.013144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fecf8-c906-42f3-85d4-6c0aff6ee48e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:24.013423 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fecf8-c906-42f3-85d4-6c0aff6ee48e None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:24.014147 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fecf8-c906-42f3-85d4-6c0aff6ee48e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:24.014505 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fecf8-c906-42f3-85d4-6c0aff6ee48e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:24.015937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fecf8-c906-42f3-85d4-6c0aff6ee48e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:24.026620 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 500/1002] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:31:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:24.027526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fecf8-c906-42f3-85d4-6c0aff6ee48e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:24.063079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d4673-b5fb-4604-9334-39a35f357ecf None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:24.073645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:24.107931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e128ac6-7435-456b-809a-4291d6a06da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:24.109351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e128ac6-7435-456b-809a-4291d6a06da7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:24.109351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e128ac6-7435-456b-809a-4291d6a06da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:24.109540 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e128ac6-7435-456b-809a-4291d6a06da7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:24.114847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e128ac6-7435-456b-809a-4291d6a06da7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:24.114847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e128ac6-7435-456b-809a-4291d6a06da7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-0e128ac6-7435-456b-809a-4291d6a06da7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:24.124478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:24.126436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:24.127514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 503/1003] 10.222.0.88 () {56 vars in 1105 bytes} [Mon Jul 18 16:31:23 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:24.132716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e128ac6-7435-456b-809a-4291d6a06da7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:24.150293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fecf8-c906-42f3-85d4-6c0aff6ee48e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:24.188220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94ea45-a3d0-476e-8e73-36e841b896af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:24.188988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ea45-a3d0-476e-8e73-36e841b896af None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:24.189275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ea45-a3d0-476e-8e73-36e841b896af None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:24.189556 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94ea45-a3d0-476e-8e73-36e841b896af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:24.190352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94ea45-a3d0-476e-8e73-36e841b896af None admin] RBAC: Adding query filter params (interface=public, service_id=9994f1b049624ba3977fff9578627ba7, region_id=RegionOne) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:24.190664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94ea45-a3d0-476e-8e73-36e841b896af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:24.192040 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94ea45-a3d0-476e-8e73-36e841b896af None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:24.197693 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 501/1004] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Jul 18 16:31:24 2022] GET /identity/v3/endpoints?service_id=9994f1b049624ba3977fff9578627ba7&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:24.202958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94ea45-a3d0-476e-8e73-36e841b896af None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:24.244939 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e128ac6-7435-456b-809a-4291d6a06da7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:24.262844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:24.285163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e67a8-1744-4d4a-b634-a0208901195d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:24.285944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e67a8-1744-4d4a-b634-a0208901195d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:24.286524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e67a8-1744-4d4a-b634-a0208901195d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:24.286637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e67a8-1744-4d4a-b634-a0208901195d None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:24.287421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e67a8-1744-4d4a-b634-a0208901195d None admin] RBAC: Adding query filter params (name=image) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:24.287735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e67a8-1744-4d4a-b634-a0208901195d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:24.289187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e67a8-1744-4d4a-b634-a0208901195d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:24.295365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 504/1005] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:24.301379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e67a8-1744-4d4a-b634-a0208901195d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:24.310689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94ea45-a3d0-476e-8e73-36e841b896af None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:24.312507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:24.351318 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc7d79-2965-46f2-b945-462336a3b0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:24.352060 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc7d79-2965-46f2-b945-462336a3b0dd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:24.352347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc7d79-2965-46f2-b945-462336a3b0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:24.352626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc7d79-2965-46f2-b945-462336a3b0dd None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:24.353357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc7d79-2965-46f2-b945-462336a3b0dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:24.354739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc7d79-2965-46f2-b945-462336a3b0dd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:24.364942 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 502/1006] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Jul 18 16:31:24 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:31:24.434037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e67a8-1744-4d4a-b634-a0208901195d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:24.472300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f634fd-e881-48b6-a6e1-ee3ad4cb1180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:24.473013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f634fd-e881-48b6-a6e1-ee3ad4cb1180 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:24.473301 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f634fd-e881-48b6-a6e1-ee3ad4cb1180 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:24.473581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f634fd-e881-48b6-a6e1-ee3ad4cb1180 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:24.474308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f634fd-e881-48b6-a6e1-ee3ad4cb1180 None admin] RBAC: Adding query filter params (type=image) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:24.474617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f634fd-e881-48b6-a6e1-ee3ad4cb1180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:24.476024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f634fd-e881-48b6-a6e1-ee3ad4cb1180 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:24.488768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 505/1007] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:24 2022] GET /identity/v3/services?type=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:25.988086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a89ad-8d40-4474-b825-e6965b7ecb97 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:25.988629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a89ad-8d40-4474-b825-e6965b7ecb97 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:25.988629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a89ad-8d40-4474-b825-e6965b7ecb97 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:25.992924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 503/1008] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:25.996130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2bad5-fc32-4a69-bcd7-e7d8e6b2307b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:25.996738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2bad5-fc32-4a69-bcd7-e7d8e6b2307b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:25.996738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2bad5-fc32-4a69-bcd7-e7d8e6b2307b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:26.014050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e9e2bad5-fc32-4a69-bcd7-e7d8e6b2307b None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:26.015247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e9e2bad5-fc32-4a69-bcd7-e7d8e6b2307b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:26.052495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:26.088861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 506/1009] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:25 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:26.112674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a89ad-8d40-4474-b825-e6965b7ecb97 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:26.220247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a89ad-8d40-4474-b825-e6965b7ecb97 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:26.256699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67331544-1a31-4e0d-a8af-eae0b4083ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:26.257537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67331544-1a31-4e0d-a8af-eae0b4083ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:26.257816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67331544-1a31-4e0d-a8af-eae0b4083ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:26.258084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67331544-1a31-4e0d-a8af-eae0b4083ef7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:26.259271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67331544-1a31-4e0d-a8af-eae0b4083ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:26.260702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67331544-1a31-4e0d-a8af-eae0b4083ef7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:26.263102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 504/1010] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:26.268669 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2bad5-fc32-4a69-bcd7-e7d8e6b2307b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:26.369725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2bad5-fc32-4a69-bcd7-e7d8e6b2307b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:26.400529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a403323-a2ea-4aa3-bb6f-3775637971d4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:26.400814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a403323-a2ea-4aa3-bb6f-3775637971d4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:26.401129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a403323-a2ea-4aa3-bb6f-3775637971d4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:26.403147 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 505/1011] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:26.409127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7886bc-56ab-4684-8959-de601212d94a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:26.409391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7886bc-56ab-4684-8959-de601212d94a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:26.409870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7886bc-56ab-4684-8959-de601212d94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:26.412701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f669f8e-97e4-4bcc-9f70-2c54d2714f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:26.413407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f669f8e-97e4-4bcc-9f70-2c54d2714f03 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:26.413690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f669f8e-97e4-4bcc-9f70-2c54d2714f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:26.413963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f669f8e-97e4-4bcc-9f70-2c54d2714f03 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:26.414694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f669f8e-97e4-4bcc-9f70-2c54d2714f03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:26.416144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f669f8e-97e4-4bcc-9f70-2c54d2714f03 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:26.428654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ec7886bc-56ab-4684-8959-de601212d94a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:26.430393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ec7886bc-56ab-4684-8959-de601212d94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:26.459130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:26.499437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 506/1012] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:26.520932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 507/1013] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:31:26 2022] POST /identity/v3/projects => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:26.526701 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7886bc-56ab-4684-8959-de601212d94a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:26.554781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb04dd3-d367-42ba-b5d9-b33d7d9bf4f7 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:26.555135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb04dd3-d367-42ba-b5d9-b33d7d9bf4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:26.555474 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb04dd3-d367-42ba-b5d9-b33d7d9bf4f7 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:26.557004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 508/1014] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:26.563648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58634b93-aa16-4668-b163-48fbd1d24920 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:26.564021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58634b93-aa16-4668-b163-48fbd1d24920 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:26.564231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58634b93-aa16-4668-b163-48fbd1d24920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:26.584602 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-58634b93-aa16-4668-b163-48fbd1d24920 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:26.585977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-58634b93-aa16-4668-b163-48fbd1d24920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:26.608704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:26.647097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7886bc-56ab-4684-8959-de601212d94a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:26.671916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 509/1015] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:26.689162 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d8f27-a36c-480a-a530-26d72d417197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:26.690041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d8f27-a36c-480a-a530-26d72d417197 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:26.690338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d8f27-a36c-480a-a530-26d72d417197 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:26.690611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d8f27-a36c-480a-a530-26d72d417197 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:26.691278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d8f27-a36c-480a-a530-26d72d417197 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:26.692905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d8f27-a36c-480a-a530-26d72d417197 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:26.693816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58634b93-aa16-4668-b163-48fbd1d24920 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-012d8f27-a36c-480a-a530-26d72d417197 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:26.709074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:26.710269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:26.711104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 507/1016] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:26.716247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d8f27-a36c-480a-a530-26d72d417197 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:26.748182 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:26.809745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58634b93-aa16-4668-b163-48fbd1d24920 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:26.834166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d8f27-a36c-480a-a530-26d72d417197 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:26.847510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abb992-994e-4090-a86a-1b40dbe2be16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:26.848240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abb992-994e-4090-a86a-1b40dbe2be16 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:26.848533 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abb992-994e-4090-a86a-1b40dbe2be16 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:26.848811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24abb992-994e-4090-a86a-1b40dbe2be16 None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:26.849605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abb992-994e-4090-a86a-1b40dbe2be16 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:26.850997 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24abb992-994e-4090-a86a-1b40dbe2be16 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:26.871709 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb7ee-3010-4238-9e76-1bd621af887f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:26.872377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb7ee-3010-4238-9e76-1bd621af887f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:26.872659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb7ee-3010-4238-9e76-1bd621af887f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:26.872965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb7ee-3010-4238-9e76-1bd621af887f None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:26.873678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fb7ee-3010-4238-9e76-1bd621af887f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:26.873988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fb7ee-3010-4238-9e76-1bd621af887f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:26.875365 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fb7ee-3010-4238-9e76-1bd621af887f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:26.880304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 508/1017] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:31:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:26.885822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb7ee-3010-4238-9e76-1bd621af887f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:26.945692 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 510/1018] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:31:26 2022] POST /identity/v3/services => generated 242 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:31:27.001830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:27.012311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb7ee-3010-4238-9e76-1bd621af887f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:27.057846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414c978-d412-4d0c-9cda-34b836df793c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:27.059565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414c978-d412-4d0c-9cda-34b836df793c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:27.060147 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414c978-d412-4d0c-9cda-34b836df793c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:27.060656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414c978-d412-4d0c-9cda-34b836df793c None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:27.061628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414c978-d412-4d0c-9cda-34b836df793c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:27.062303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414c978-d412-4d0c-9cda-34b836df793c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:27.063939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414c978-d412-4d0c-9cda-34b836df793c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:27.070546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 509/1019] 10.222.0.88 () {56 vars in 1112 bytes} [Mon Jul 18 16:31:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:27.076256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abb992-994e-4090-a86a-1b40dbe2be16 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:27.095289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:27.191517 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24abb992-994e-4090-a86a-1b40dbe2be16 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:27.229464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d7a26-fec3-42b7-bef5-f1aa7025c3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:27.230690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d7a26-fec3-42b7-bef5-f1aa7025c3de None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:27.230976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d7a26-fec3-42b7-bef5-f1aa7025c3de None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:27.231254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d7a26-fec3-42b7-bef5-f1aa7025c3de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:27.232020 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d7a26-fec3-42b7-bef5-f1aa7025c3de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:27.233518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d7a26-fec3-42b7-bef5-f1aa7025c3de None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:27.348354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 511/1020] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Jul 18 16:31:27 2022] POST /identity/v3/endpoints => generated 334 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:28.738682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a85847-0f06-4469-a36d-141ce22bf3de None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:28.739210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a85847-0f06-4469-a36d-141ce22bf3de None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:28.739517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a85847-0f06-4469-a36d-141ce22bf3de None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:28.742103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 510/1021] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:28.749773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9008c2-4a51-4aab-98b2-454275f83177 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:28.750400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9008c2-4a51-4aab-98b2-454275f83177 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:28.750400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9008c2-4a51-4aab-98b2-454275f83177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:28.768768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3c9008c2-4a51-4aab-98b2-454275f83177 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:28.770492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3c9008c2-4a51-4aab-98b2-454275f83177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:28.836417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:28.848310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 512/1022] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:28.873331 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a85847-0f06-4469-a36d-141ce22bf3de None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:28.991074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a85847-0f06-4469-a36d-141ce22bf3de None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:29.026545 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2abac9-dab8-4557-8c77-b642d6ed01d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:29.027399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2abac9-dab8-4557-8c77-b642d6ed01d4 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.027682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2abac9-dab8-4557-8c77-b642d6ed01d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.027959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2abac9-dab8-4557-8c77-b642d6ed01d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.029218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2abac9-dab8-4557-8c77-b642d6ed01d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:29.030671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2abac9-dab8-4557-8c77-b642d6ed01d4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:29.032473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 511/1023] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:29.038406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9008c2-4a51-4aab-98b2-454275f83177 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:29.063356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bec8c1-71b9-47f3-b4d9-eb755c1d56c8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.063642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bec8c1-71b9-47f3-b4d9-eb755c1d56c8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.063924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bec8c1-71b9-47f3-b4d9-eb755c1d56c8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.065510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 512/1024] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:29.070913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41ab44-f31b-4c8d-9d7b-d8375d789925 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.071500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41ab44-f31b-4c8d-9d7b-d8375d789925 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.071500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41ab44-f31b-4c8d-9d7b-d8375d789925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.090216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bd41ab44-f31b-4c8d-9d7b-d8375d789925 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:29.091416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bd41ab44-f31b-4c8d-9d7b-d8375d789925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:29.107009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:29.153182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9008c2-4a51-4aab-98b2-454275f83177 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:29.163785 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 513/1025] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:29.186685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41ab44-f31b-4c8d-9d7b-d8375d789925 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:29.190773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173ba1c-e4fd-4e81-9802-ddcdc39b3464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:29.191708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173ba1c-e4fd-4e81-9802-ddcdc39b3464 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.192135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173ba1c-e4fd-4e81-9802-ddcdc39b3464 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.192568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173ba1c-e4fd-4e81-9802-ddcdc39b3464 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.193461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173ba1c-e4fd-4e81-9802-ddcdc39b3464 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:29.195060 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173ba1c-e4fd-4e81-9802-ddcdc39b3464 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:29.288306 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 513/1026] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:31:29 2022] POST /identity/v3/users => generated 285 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:29.288821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41ab44-f31b-4c8d-9d7b-d8375d789925 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:29.337037 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe457fd-0127-4e7d-a203-157194f45ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:29.337857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe457fd-0127-4e7d-a203-157194f45ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.338140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe457fd-0127-4e7d-a203-157194f45ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.338414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe457fd-0127-4e7d-a203-157194f45ca7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.339093 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe457fd-0127-4e7d-a203-157194f45ca7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:29.341174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe457fd-0127-4e7d-a203-157194f45ca7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-bbe457fd-0127-4e7d-a203-157194f45ca7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:29.353979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:29.355101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:29.355954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 514/1027] 10.222.0.88 () {56 vars in 1105 bytes} [Mon Jul 18 16:31:29 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:29.365001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173ba1c-e4fd-4e81-9802-ddcdc39b3464 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:29.428214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:29.480334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173ba1c-e4fd-4e81-9802-ddcdc39b3464 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:29.482810 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753a511-eaad-4475-96fb-97b2c82881af None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.483099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753a511-eaad-4475-96fb-97b2c82881af None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.483383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753a511-eaad-4475-96fb-97b2c82881af None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.484771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 515/1028] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:29.489879 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2f244-8a7b-457c-bc8c-3073bda65f4c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.490176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2f244-8a7b-457c-bc8c-3073bda65f4c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.490462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2f244-8a7b-457c-bc8c-3073bda65f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.509597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-37f2f244-8a7b-457c-bc8c-3073bda65f4c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:29.510858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-37f2f244-8a7b-457c-bc8c-3073bda65f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:29.519197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1644e7-f3d3-420c-ba24-0423b066dcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:29.519869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1644e7-f3d3-420c-ba24-0423b066dcbd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.520155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1644e7-f3d3-420c-ba24-0423b066dcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.520432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1644e7-f3d3-420c-ba24-0423b066dcbd None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.521733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1644e7-f3d3-420c-ba24-0423b066dcbd None admin] RBAC: Adding query filter params (name=image) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:29.521733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1644e7-f3d3-420c-ba24-0423b066dcbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:29.521992 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:29.523842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1644e7-f3d3-420c-ba24-0423b066dcbd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:29.531184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 514/1029] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:29.537187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1644e7-f3d3-420c-ba24-0423b066dcbd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:29.598535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 516/1030] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:29.621835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2f244-8a7b-457c-bc8c-3073bda65f4c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:29.652048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1644e7-f3d3-420c-ba24-0423b066dcbd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:29.670711 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:29.693369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea47ab-c915-4a6d-a6a8-39a8422c367d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:29.694173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea47ab-c915-4a6d-a6a8-39a8422c367d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.694391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea47ab-c915-4a6d-a6a8-39a8422c367d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.694622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea47ab-c915-4a6d-a6a8-39a8422c367d None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.695346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea47ab-c915-4a6d-a6a8-39a8422c367d None admin] RBAC: Adding query filter params (type=image) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:29.695663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea47ab-c915-4a6d-a6a8-39a8422c367d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:29.697115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea47ab-c915-4a6d-a6a8-39a8422c367d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:29.709698 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 515/1031] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:29.711072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea47ab-c915-4a6d-a6a8-39a8422c367d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:29.731381 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2f244-8a7b-457c-bc8c-3073bda65f4c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:29.771342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04cfd7-bf43-4388-bd1c-cd364db8c73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:29.772281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04cfd7-bf43-4388-bd1c-cd364db8c73a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.772567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04cfd7-bf43-4388-bd1c-cd364db8c73a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.772837 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04cfd7-bf43-4388-bd1c-cd364db8c73a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.784766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04cfd7-bf43-4388-bd1c-cd364db8c73a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4d04cfd7-bf43-4388-bd1c-cd364db8c73a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:29.793671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:29.794628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:29.795498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 517/1032] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Jul 18 16:31:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:29.800710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04cfd7-bf43-4388-bd1c-cd364db8c73a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:29.805592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:29.826540 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea47ab-c915-4a6d-a6a8-39a8422c367d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:29.865346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c30eb3-149c-4a89-bc4e-a19c8c8a8f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:29.865928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c30eb3-149c-4a89-bc4e-a19c8c8a8f96 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.866212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c30eb3-149c-4a89-bc4e-a19c8c8a8f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.866808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c30eb3-149c-4a89-bc4e-a19c8c8a8f96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.867693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c30eb3-149c-4a89-bc4e-a19c8c8a8f96 None admin] RBAC: Adding query filter params (interface=public, service_id=9d5259103e194622aabc49129d8869ba, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:29.868083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c30eb3-149c-4a89-bc4e-a19c8c8a8f96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:29.869686 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c30eb3-149c-4a89-bc4e-a19c8c8a8f96 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:29.880741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 516/1033] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Jul 18 16:31:29 2022] GET /identity/v3/endpoints?service_id=9d5259103e194622aabc49129d8869ba&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:29.881740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c30eb3-149c-4a89-bc4e-a19c8c8a8f96 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:29.918366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04cfd7-bf43-4388-bd1c-cd364db8c73a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:29.937242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:29.969640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf86ea1-d10d-4aec-b879-24143f517dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:29.973391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf86ea1-d10d-4aec-b879-24143f517dff None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:29.973391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf86ea1-d10d-4aec-b879-24143f517dff None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:29.973391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf86ea1-d10d-4aec-b879-24143f517dff None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:29.973391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf86ea1-d10d-4aec-b879-24143f517dff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:29.973391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf86ea1-d10d-4aec-b879-24143f517dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:29.973391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf86ea1-d10d-4aec-b879-24143f517dff None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:29.982791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 518/1034] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Jul 18 16:31:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:29.988354 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf86ea1-d10d-4aec-b879-24143f517dff None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:30.000821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c30eb3-149c-4a89-bc4e-a19c8c8a8f96 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:30.038384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23540750-95d7-4fa6-a1ae-a0f4cad7178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:30.039290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23540750-95d7-4fa6-a1ae-a0f4cad7178c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:30.039413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23540750-95d7-4fa6-a1ae-a0f4cad7178c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:30.039628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23540750-95d7-4fa6-a1ae-a0f4cad7178c None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:30.040315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23540750-95d7-4fa6-a1ae-a0f4cad7178c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:30.041712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23540750-95d7-4fa6-a1ae-a0f4cad7178c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:30.049135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 517/1035] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Jul 18 16:31:29 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:31:30.063722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:30.104428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf86ea1-d10d-4aec-b879-24143f517dff None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:30.143966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db66342-3e0a-414f-8f9b-7baf337048ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:30.144557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db66342-3e0a-414f-8f9b-7baf337048ef None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:30.144841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db66342-3e0a-414f-8f9b-7baf337048ef None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:30.145150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db66342-3e0a-414f-8f9b-7baf337048ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:30.157953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db66342-3e0a-414f-8f9b-7baf337048ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:30.159445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db66342-3e0a-414f-8f9b-7baf337048ef None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-0db66342-3e0a-414f-8f9b-7baf337048ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:30.178808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:30.179898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:30.180873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:30.180873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:30.180873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:30.180873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:30.180873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 519/1036] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:30.186627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23540750-95d7-4fa6-a1ae-a0f4cad7178c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:30.297282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23540750-95d7-4fa6-a1ae-a0f4cad7178c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:30.306597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:30.333027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238a4cc-af55-4424-a288-a7722439848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:30.333704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238a4cc-af55-4424-a288-a7722439848b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:30.333989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238a4cc-af55-4424-a288-a7722439848b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:30.334263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238a4cc-af55-4424-a288-a7722439848b None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:30.334987 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238a4cc-af55-4424-a288-a7722439848b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:30.335293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238a4cc-af55-4424-a288-a7722439848b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:30.336668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0238a4cc-af55-4424-a288-a7722439848b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:30.348860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 518/1037] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:30.354245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db66342-3e0a-414f-8f9b-7baf337048ef None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:30.461170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db66342-3e0a-414f-8f9b-7baf337048ef None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:30.478883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:30.497256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a0857-57cd-497e-8551-3fb1feff1e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:30.504510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a0857-57cd-497e-8551-3fb1feff1e81 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:30.504510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a0857-57cd-497e-8551-3fb1feff1e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:30.504510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a0857-57cd-497e-8551-3fb1feff1e81 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:30.507379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a0857-57cd-497e-8551-3fb1feff1e81 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:30.508903 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a0857-57cd-497e-8551-3fb1feff1e81 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-878a0857-57cd-497e-8551-3fb1feff1e81 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:31:30.524710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:30.525774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:30.527228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:30.527228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:30.527228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:31:30.527228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:31:30.527228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:31:30.527228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:31:30.527228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:31:30.527228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:31:30.527228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:31:30.527228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:30.532427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 520/1038] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Jul 18 16:31:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:30.532584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238a4cc-af55-4424-a288-a7722439848b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:30.642621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238a4cc-af55-4424-a288-a7722439848b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:30.676168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114a20e-078d-4051-91ec-88daf26c19f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:30.676910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114a20e-078d-4051-91ec-88daf26c19f1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:30.677215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114a20e-078d-4051-91ec-88daf26c19f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:30.677487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114a20e-078d-4051-91ec-88daf26c19f1 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:30.678314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114a20e-078d-4051-91ec-88daf26c19f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:30.678616 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114a20e-078d-4051-91ec-88daf26c19f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:30.680031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114a20e-078d-4051-91ec-88daf26c19f1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:30.698213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 519/1039] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Jul 18 16:31:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:30.704244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a0857-57cd-497e-8551-3fb1feff1e81 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:30.806321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a0857-57cd-497e-8551-3fb1feff1e81 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:30.841307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80c5cf-eee7-4d6b-996b-08e89df0cabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:30.841959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c5cf-eee7-4d6b-996b-08e89df0cabd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:30.842239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c5cf-eee7-4d6b-996b-08e89df0cabd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:30.842514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80c5cf-eee7-4d6b-996b-08e89df0cabd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:30.890055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:30.912067 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80c5cf-eee7-4d6b-996b-08e89df0cabd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:30.913555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80c5cf-eee7-4d6b-996b-08e89df0cabd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:31.024562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 521/1040] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:31:31.025872 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114a20e-078d-4051-91ec-88daf26c19f1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:31.054806 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:31.135186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114a20e-078d-4051-91ec-88daf26c19f1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:31.172405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b1569-dacd-486a-b853-37cc516d5862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:31.173286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b1569-dacd-486a-b853-37cc516d5862 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.173569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b1569-dacd-486a-b853-37cc516d5862 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.173848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b1569-dacd-486a-b853-37cc516d5862 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.186385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b1569-dacd-486a-b853-37cc516d5862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:31.187864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b1569-dacd-486a-b853-37cc516d5862 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-1e6b1569-dacd-486a-b853-37cc516d5862 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:31.207473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:31.208450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:31.209410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:31.209410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:31.209410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:31.209410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:31.213226 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 520/1041] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:31.214512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80c5cf-eee7-4d6b-996b-08e89df0cabd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:31.297428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:31.320796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80c5cf-eee7-4d6b-996b-08e89df0cabd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:31.357036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fdcf2-7725-4205-9ab8-464b5521f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:31.357700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fdcf2-7725-4205-9ab8-464b5521f5ea None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.357985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fdcf2-7725-4205-9ab8-464b5521f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.358261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fdcf2-7725-4205-9ab8-464b5521f5ea None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.358982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fdcf2-7725-4205-9ab8-464b5521f5ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:31.359291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fdcf2-7725-4205-9ab8-464b5521f5ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:31.360663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fdcf2-7725-4205-9ab8-464b5521f5ea None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:31.372742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 522/1042] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:31.378169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b1569-dacd-486a-b853-37cc516d5862 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:31.436655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50516ccc-3c8d-47cb-a9ff-94706738f9ce None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.436998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50516ccc-3c8d-47cb-a9ff-94706738f9ce None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.437262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50516ccc-3c8d-47cb-a9ff-94706738f9ce None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.438687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 523/1043] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:31.443584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c441b-0de5-49dc-99e1-6845d032d035 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.443875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c441b-0de5-49dc-99e1-6845d032d035 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.444151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c441b-0de5-49dc-99e1-6845d032d035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.461017 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d93c441b-0de5-49dc-99e1-6845d032d035 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:31.462210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d93c441b-0de5-49dc-99e1-6845d032d035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:31.465390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:31.488639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b1569-dacd-486a-b853-37cc516d5862 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:31.524901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ce4e0-18d0-44d0-8c0a-61e03c9eb380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:31.525721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ce4e0-18d0-44d0-8c0a-61e03c9eb380 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.526065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ce4e0-18d0-44d0-8c0a-61e03c9eb380 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.526289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ce4e0-18d0-44d0-8c0a-61e03c9eb380 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.533847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ce4e0-18d0-44d0-8c0a-61e03c9eb380 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:31.535299 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ce4e0-18d0-44d0-8c0a-61e03c9eb380 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:31.537535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 524/1044] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-e12ce4e0-18d0-44d0-8c0a-61e03c9eb380 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:31:31.550102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:31.551033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:31.551978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:31:31.551978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:31:31.551978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:31:31.551978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:31:31.551978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:31:31.551978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:31:31.551978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:31.551978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:31.551978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 521/1045] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:31.559170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c441b-0de5-49dc-99e1-6845d032d035 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:31.561384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ce4e0-18d0-44d0-8c0a-61e03c9eb380 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:31.630632 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:31.665775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c441b-0de5-49dc-99e1-6845d032d035 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:31.666385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ce4e0-18d0-44d0-8c0a-61e03c9eb380 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:31.703414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a12c3e-01e6-46d5-b11e-6496ac1a9a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:31.703794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89083a-7da1-421b-9a17-9ab648e95471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:31.704135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a12c3e-01e6-46d5-b11e-6496ac1a9a5c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.704419 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a12c3e-01e6-46d5-b11e-6496ac1a9a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.704526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89083a-7da1-421b-9a17-9ab648e95471 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.704690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a12c3e-01e6-46d5-b11e-6496ac1a9a5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.704793 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89083a-7da1-421b-9a17-9ab648e95471 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.705047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89083a-7da1-421b-9a17-9ab648e95471 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.705844 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89083a-7da1-421b-9a17-9ab648e95471 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:31.706155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89083a-7da1-421b-9a17-9ab648e95471 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:31.707606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89083a-7da1-421b-9a17-9ab648e95471 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:31.712464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:31.717569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a12c3e-01e6-46d5-b11e-6496ac1a9a5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:31.719914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 525/1046] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:31:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a1a12c3e-01e6-46d5-b11e-6496ac1a9a5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:31.725572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:31.733149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:31:31.734024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:31.734768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89083a-7da1-421b-9a17-9ab648e95471 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:31.734853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 522/1047] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:31:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:31.734853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a12c3e-01e6-46d5-b11e-6496ac1a9a5c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:31.835902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89083a-7da1-421b-9a17-9ab648e95471 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:31.846812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a12c3e-01e6-46d5-b11e-6496ac1a9a5c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:31.865148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:31.873726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f8aec-c3b0-49c8-a0ce-bf0a3b27c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:31.874436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f8aec-c3b0-49c8-a0ce-bf0a3b27c45f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.874749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f8aec-c3b0-49c8-a0ce-bf0a3b27c45f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.874986 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f8aec-c3b0-49c8-a0ce-bf0a3b27c45f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.875819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f8aec-c3b0-49c8-a0ce-bf0a3b27c45f None admin] RBAC: Adding query filter params (role.id=443cff586282466c87ffce62f1440b63, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=69ee145599384c6da2cba85a42832dcd) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:31.876125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f8aec-c3b0-49c8-a0ce-bf0a3b27c45f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:31.877624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f8aec-c3b0-49c8-a0ce-bf0a3b27c45f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:31.883484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 526/1048] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:31 2022] GET /identity/v3/role_assignments?user.id=69ee145599384c6da2cba85a42832dcd&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=443cff586282466c87ffce62f1440b63 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:31.888465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500bb85e-f125-41ee-b5aa-7e22c596edd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:31.889409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bb85e-f125-41ee-b5aa-7e22c596edd0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:31.889702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bb85e-f125-41ee-b5aa-7e22c596edd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:31.889985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bb85e-f125-41ee-b5aa-7e22c596edd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:31.890770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bb85e-f125-41ee-b5aa-7e22c596edd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:31.891086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bb85e-f125-41ee-b5aa-7e22c596edd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:31.892526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500bb85e-f125-41ee-b5aa-7e22c596edd0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:31.903532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 523/1049] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:31:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:31.909408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f8aec-c3b0-49c8-a0ce-bf0a3b27c45f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:32.038567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f8aec-c3b0-49c8-a0ce-bf0a3b27c45f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:32.075890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac82a39-dc9c-457c-a5f7-3376461d4c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:32.077013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac82a39-dc9c-457c-a5f7-3376461d4c33 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:32.077460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac82a39-dc9c-457c-a5f7-3376461d4c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:32.077835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac82a39-dc9c-457c-a5f7-3376461d4c33 None admin] PATH_INFO: `/v3/users/6013afe9b524409b83b13ea700b832a0` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:32.097459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac82a39-dc9c-457c-a5f7-3376461d4c33 None admin] RBAC: Authorizing `identity:get_user(user_id=6013afe9b524409b83b13ea700b832a0)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:32.098843 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851a3db-e533-49a3-835e-b5db5272ab00 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:32.099278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac82a39-dc9c-457c-a5f7-3376461d4c33 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:32.099372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851a3db-e533-49a3-835e-b5db5272ab00 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:32.099457 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851a3db-e533-49a3-835e-b5db5272ab00 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:32.100787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 527/1050] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:31:31 2022] GET /identity/v3/users/6013afe9b524409b83b13ea700b832a0 => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:32.105607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 524/1051] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:32.106454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851a3db-e533-49a3-835e-b5db5272ab00 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:32.106579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b06a1-eb33-4630-b279-77e97f8340b0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:32.106790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b06a1-eb33-4630-b279-77e97f8340b0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:32.107316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b06a1-eb33-4630-b279-77e97f8340b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:32.128489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7f7b06a1-eb33-4630-b279-77e97f8340b0 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:32.129888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7f7b06a1-eb33-4630-b279-77e97f8340b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:32.183637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:32.211172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 528/1052] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:32.231905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:32.249205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b06a1-eb33-4630-b279-77e97f8340b0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:32.287768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851a3db-e533-49a3-835e-b5db5272ab00 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:32.330024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe155-4c0e-4356-a551-c677170cbf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:32.330606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe155-4c0e-4356-a551-c677170cbf14 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:32.330891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe155-4c0e-4356-a551-c677170cbf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:32.331172 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe155-4c0e-4356-a551-c677170cbf14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:32.384597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b06a1-eb33-4630-b279-77e97f8340b0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:32.437681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fe155-4c0e-4356-a551-c677170cbf14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:32.438491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735819a-20e5-49ff-99a7-062b870ac59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:32.439154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fe155-4c0e-4356-a551-c677170cbf14 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:32.439886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735819a-20e5-49ff-99a7-062b870ac59a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:32.440792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735819a-20e5-49ff-99a7-062b870ac59a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:32.442025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735819a-20e5-49ff-99a7-062b870ac59a None admin] PATH_INFO: `/v3/services/image` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:32.444681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735819a-20e5-49ff-99a7-062b870ac59a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:32.446621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735819a-20e5-49ff-99a7-062b870ac59a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8735819a-20e5-49ff-99a7-062b870ac59a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:32.462454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:32.463568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:32.465086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 529/1053] 10.222.0.88 () {56 vars in 1105 bytes} [Mon Jul 18 16:31:32 2022] GET /identity/v3/services/image => generated 86 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:32.470167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735819a-20e5-49ff-99a7-062b870ac59a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:32.512499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:32.522329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:32.560133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 525/1054] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:32 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:31:32.561657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe155-4c0e-4356-a551-c677170cbf14 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:32.580270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735819a-20e5-49ff-99a7-062b870ac59a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:32.617024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ee763-22a3-45f0-b809-913c90470172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:32.617697 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ee763-22a3-45f0-b809-913c90470172 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:32.617975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ee763-22a3-45f0-b809-913c90470172 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:32.618247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ee763-22a3-45f0-b809-913c90470172 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:32.618959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ee763-22a3-45f0-b809-913c90470172 None admin] RBAC: Adding query filter params (name=image) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:32.619264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ee763-22a3-45f0-b809-913c90470172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:32.620729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ee763-22a3-45f0-b809-913c90470172 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:32.626204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 530/1055] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:32.632340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ee763-22a3-45f0-b809-913c90470172 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:32.675529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe155-4c0e-4356-a551-c677170cbf14 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:32.709834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b62edc-2043-455c-919a-192c92a92b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:32.710738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62edc-2043-455c-919a-192c92a92b6b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:32.711028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62edc-2043-455c-919a-192c92a92b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:32.711297 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b62edc-2043-455c-919a-192c92a92b6b None admin] PATH_INFO: `/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:32.712465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b62edc-2043-455c-919a-192c92a92b6b None admin] RBAC: Authorizing `identity:get_project(project_id=e3dfff089a14434d896cbd3f4ed0ea45)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:32.713930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b62edc-2043-455c-919a-192c92a92b6b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:32.716315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 526/1056] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:31:32 2022] GET /identity/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:32.722038 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b62edc-2043-455c-919a-192c92a92b6b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:32.746901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ee763-22a3-45f0-b809-913c90470172 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:32.751046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:32.757685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:32.784417 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e279d-3b39-48d5-8e62-2476524a6f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:32.785262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e279d-3b39-48d5-8e62-2476524a6f32 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:32.785612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e279d-3b39-48d5-8e62-2476524a6f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:32.785948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e279d-3b39-48d5-8e62-2476524a6f32 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:32.786662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e279d-3b39-48d5-8e62-2476524a6f32 None admin] RBAC: Adding query filter params (type=image) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:32.786966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e279d-3b39-48d5-8e62-2476524a6f32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:32.788345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e279d-3b39-48d5-8e62-2476524a6f32 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:32.793957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 531/1057] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:32.799402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e279d-3b39-48d5-8e62-2476524a6f32 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:32.829218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b62edc-2043-455c-919a-192c92a92b6b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:32.865305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc5006-fa41-43eb-8c1c-9e5f4db4f8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:32.865879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc5006-fa41-43eb-8c1c-9e5f4db4f8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:32.866161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc5006-fa41-43eb-8c1c-9e5f4db4f8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:32.866689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc5006-fa41-43eb-8c1c-9e5f4db4f8f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:32.867531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc5006-fa41-43eb-8c1c-9e5f4db4f8f1 None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=e3dfff089a14434d896cbd3f4ed0ea45, user.id=6013afe9b524409b83b13ea700b832a0) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:32.867841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc5006-fa41-43eb-8c1c-9e5f4db4f8f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:32.869292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc5006-fa41-43eb-8c1c-9e5f4db4f8f1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:32.884912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 527/1058] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:32 2022] GET /identity/v3/role_assignments?user.id=6013afe9b524409b83b13ea700b832a0&scope.project.id=e3dfff089a14434d896cbd3f4ed0ea45&role.id=81457691f3a2456fa33358cb6a922d05 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:32.918608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e279d-3b39-48d5-8e62-2476524a6f32 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:32.958369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630c369-2a95-4735-98fb-4ae7d9c691da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:32.959041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630c369-2a95-4735-98fb-4ae7d9c691da None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:32.959319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630c369-2a95-4735-98fb-4ae7d9c691da None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:32.959594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630c369-2a95-4735-98fb-4ae7d9c691da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:32.960348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630c369-2a95-4735-98fb-4ae7d9c691da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:32.961814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630c369-2a95-4735-98fb-4ae7d9c691da None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:33.070026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 532/1059] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Jul 18 16:31:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:31:33.935709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13485301-69d0-4f4f-93ca-90a2fb5bc822 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:33.936135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13485301-69d0-4f4f-93ca-90a2fb5bc822 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:33.936252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13485301-69d0-4f4f-93ca-90a2fb5bc822 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:33.938055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 528/1060] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:33.943221 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3e2f2-e324-45ae-9047-fad206415dba None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:33.943651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3e2f2-e324-45ae-9047-fad206415dba None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:33.943822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3e2f2-e324-45ae-9047-fad206415dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:33.961523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e5d3e2f2-e324-45ae-9047-fad206415dba None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:33.962749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e5d3e2f2-e324-45ae-9047-fad206415dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:33.986989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:34.042037 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 533/1061] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:34.064643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13485301-69d0-4f4f-93ca-90a2fb5bc822 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:34.178282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:34.188638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13485301-69d0-4f4f-93ca-90a2fb5bc822 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:34.226804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ade07d-b2e0-454e-bb76-8d27667c1182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:34.227621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade07d-b2e0-454e-bb76-8d27667c1182 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:34.227909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade07d-b2e0-454e-bb76-8d27667c1182 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:34.228182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade07d-b2e0-454e-bb76-8d27667c1182 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:34.238551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ade07d-b2e0-454e-bb76-8d27667c1182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-67ade07d-b2e0-454e-bb76-8d27667c1182 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:34.247530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:34.248543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:34.249432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:34.250697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 529/1062] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Jul 18 16:31:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:34.254724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3e2f2-e324-45ae-9047-fad206415dba None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:34.360256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3e2f2-e324-45ae-9047-fad206415dba None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:34.395452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb6be5-388b-4729-9ad9-b3d97753d4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:34.396120 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb6be5-388b-4729-9ad9-b3d97753d4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:34.396402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb6be5-388b-4729-9ad9-b3d97753d4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:34.396674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb6be5-388b-4729-9ad9-b3d97753d4a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:34.397452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb6be5-388b-4729-9ad9-b3d97753d4a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:34.397761 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb6be5-388b-4729-9ad9-b3d97753d4a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:34.399136 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb6be5-388b-4729-9ad9-b3d97753d4a2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:34.412932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 534/1063] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Jul 18 16:31:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:34.414148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ade07d-b2e0-454e-bb76-8d27667c1182 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:34.518603 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ade07d-b2e0-454e-bb76-8d27667c1182 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:34.555437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2be9a-b762-40a1-8fd1-35ac87780872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:34.556238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2be9a-b762-40a1-8fd1-35ac87780872 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:34.556522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2be9a-b762-40a1-8fd1-35ac87780872 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:34.556799 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2be9a-b762-40a1-8fd1-35ac87780872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:34.567918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2be9a-b762-40a1-8fd1-35ac87780872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:34.569430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b2be9a-b762-40a1-8fd1-35ac87780872 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b7b2be9a-b762-40a1-8fd1-35ac87780872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:34.588102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:34.589132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:34.590647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:34.590647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:34.590647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:34.590647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:34.598211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 530/1064] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:34.598412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb6be5-388b-4729-9ad9-b3d97753d4a2 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:34.617863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:34.710951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb6be5-388b-4729-9ad9-b3d97753d4a2 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:34.747023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6fbb7-07bd-4ac7-9c6b-188647b619b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:34.747754 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6fbb7-07bd-4ac7-9c6b-188647b619b8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:34.748046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6fbb7-07bd-4ac7-9c6b-188647b619b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:34.748336 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6fbb7-07bd-4ac7-9c6b-188647b619b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:34.749070 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6fbb7-07bd-4ac7-9c6b-188647b619b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:34.749376 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6fbb7-07bd-4ac7-9c6b-188647b619b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:34.750771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6fbb7-07bd-4ac7-9c6b-188647b619b8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:34.768084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 535/1065] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:34.769398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2be9a-b762-40a1-8fd1-35ac87780872 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:34.773563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:34.910509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b2be9a-b762-40a1-8fd1-35ac87780872 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:34.917386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61570384-8cf5-4392-ab71-3e7b02375cfe None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:34.917702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61570384-8cf5-4392-ab71-3e7b02375cfe None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:34.918473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61570384-8cf5-4392-ab71-3e7b02375cfe None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:34.919980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 536/1066] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:34.924916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da605-cc39-4404-8727-e303fdd12374 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:34.925643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da605-cc39-4404-8727-e303fdd12374 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:34.925922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da605-cc39-4404-8727-e303fdd12374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:34.943474 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-734da605-cc39-4404-8727-e303fdd12374 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:34.944680 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-734da605-cc39-4404-8727-e303fdd12374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:34.956693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc3db5-1e1f-447a-b3e0-a5dbe0cc23f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:34.957812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc3db5-1e1f-447a-b3e0-a5dbe0cc23f4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:34.958098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc3db5-1e1f-447a-b3e0-a5dbe0cc23f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:34.958381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc3db5-1e1f-447a-b3e0-a5dbe0cc23f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:34.967816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc3db5-1e1f-447a-b3e0-a5dbe0cc23f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:34.969304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc3db5-1e1f-447a-b3e0-a5dbe0cc23f4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-03bc3db5-1e1f-447a-b3e0-a5dbe0cc23f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:31:34.984325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:34.985402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 531/1067] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Jul 18 16:31:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:34.991750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc3db5-1e1f-447a-b3e0-a5dbe0cc23f4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:35.029146 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 537/1068] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:35.043783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734da605-cc39-4404-8727-e303fdd12374 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:35.054118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:35.059888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:35.098917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc3db5-1e1f-447a-b3e0-a5dbe0cc23f4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:35.139407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd5465-4a41-4d0d-a136-2ea061f468d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:35.140085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5465-4a41-4d0d-a136-2ea061f468d9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.140382 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5465-4a41-4d0d-a136-2ea061f468d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.140664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd5465-4a41-4d0d-a136-2ea061f468d9 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.141992 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd5465-4a41-4d0d-a136-2ea061f468d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:35.142312 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd5465-4a41-4d0d-a136-2ea061f468d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:35.143741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd5465-4a41-4d0d-a136-2ea061f468d9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:35.153939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734da605-cc39-4404-8727-e303fdd12374 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:35.162064 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 532/1069] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Jul 18 16:31:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:35.163403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f537ffa-dc6b-42d2-8062-fe86ae4764e6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.163728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f537ffa-dc6b-42d2-8062-fe86ae4764e6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.163983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f537ffa-dc6b-42d2-8062-fe86ae4764e6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.167058 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 533/1070] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:35.168629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f537ffa-dc6b-42d2-8062-fe86ae4764e6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:35.190221 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92275e6c-dc58-49e7-920b-b2a37658ee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:35.191020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92275e6c-dc58-49e7-920b-b2a37658ee34 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.191303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92275e6c-dc58-49e7-920b-b2a37658ee34 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.191573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92275e6c-dc58-49e7-920b-b2a37658ee34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.202037 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92275e6c-dc58-49e7-920b-b2a37658ee34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-92275e6c-dc58-49e7-920b-b2a37658ee34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:35.209727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:35.211118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:35.211952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 538/1071] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:31:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:35.212657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5790aa-ecb0-40ab-8202-5a6126e5e41c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.212657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5790aa-ecb0-40ab-8202-5a6126e5e41c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.212787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5790aa-ecb0-40ab-8202-5a6126e5e41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.229760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0e5790aa-ecb0-40ab-8202-5a6126e5e41c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:35.231500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0e5790aa-ecb0-40ab-8202-5a6126e5e41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:35.275719 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f537ffa-dc6b-42d2-8062-fe86ae4764e6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:35.294770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:35.300343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:35.320924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 539/1072] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:35.320924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5790aa-ecb0-40ab-8202-5a6126e5e41c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:35.321212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3ce30-c374-4c53-9626-097617be2083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:35.321212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3ce30-c374-4c53-9626-097617be2083 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.321212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3ce30-c374-4c53-9626-097617be2083 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.321212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3ce30-c374-4c53-9626-097617be2083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.324428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3ce30-c374-4c53-9626-097617be2083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:35.326053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3ce30-c374-4c53-9626-097617be2083 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-59a3ce30-c374-4c53-9626-097617be2083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:35.345068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:35.346031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:35.346971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:35.346971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:35.346971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:35.346971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:35.346971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 534/1073] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:35.347262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3ce30-c374-4c53-9626-097617be2083 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:35.407309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5790aa-ecb0-40ab-8202-5a6126e5e41c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:35.441353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b2371-6b06-4965-b2d3-827133c9f435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:35.442026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b2371-6b06-4965-b2d3-827133c9f435 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.442307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b2371-6b06-4965-b2d3-827133c9f435 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.442582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b2371-6b06-4965-b2d3-827133c9f435 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.443337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b2371-6b06-4965-b2d3-827133c9f435 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:35.443650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b2371-6b06-4965-b2d3-827133c9f435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:35.445049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b2371-6b06-4965-b2d3-827133c9f435 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:35.450747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3ce30-c374-4c53-9626-097617be2083 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:35.453730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 540/1074] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:31:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:35.454521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b2371-6b06-4965-b2d3-827133c9f435 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:35.488577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f2311-bdfb-4737-9151-f4bf480dee1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:35.489350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f2311-bdfb-4737-9151-f4bf480dee1c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.489628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f2311-bdfb-4737-9151-f4bf480dee1c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.489912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f2311-bdfb-4737-9151-f4bf480dee1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.513462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:35.535270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:35.555796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b2371-6b06-4965-b2d3-827133c9f435 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:35.559886 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f2311-bdfb-4737-9151-f4bf480dee1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:35.561368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f2311-bdfb-4737-9151-f4bf480dee1c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:35.591292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993168b-0e21-4684-9a9f-7a15c9d9ba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:35.591855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993168b-0e21-4684-9a9f-7a15c9d9ba9b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.592145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993168b-0e21-4684-9a9f-7a15c9d9ba9b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.592419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993168b-0e21-4684-9a9f-7a15c9d9ba9b None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.593173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993168b-0e21-4684-9a9f-7a15c9d9ba9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:35.593479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993168b-0e21-4684-9a9f-7a15c9d9ba9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:35.594855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993168b-0e21-4684-9a9f-7a15c9d9ba9b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:35.607401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 541/1075] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:35.608216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993168b-0e21-4684-9a9f-7a15c9d9ba9b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:35.664194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 535/1076] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:31:35.664194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f2311-bdfb-4737-9151-f4bf480dee1c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:35.716069 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993168b-0e21-4684-9a9f-7a15c9d9ba9b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:35.733900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:35.740412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:35.751554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38616077-a11a-42d9-983c-868ec33a4430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:35.752050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38616077-a11a-42d9-983c-868ec33a4430 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.752336 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38616077-a11a-42d9-983c-868ec33a4430 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.752612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38616077-a11a-42d9-983c-868ec33a4430 None admin] PATH_INFO: `/v3/users/6013afe9b524409b83b13ea700b832a0` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.753860 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38616077-a11a-42d9-983c-868ec33a4430 None admin] RBAC: Authorizing `identity:get_user(user_id=6013afe9b524409b83b13ea700b832a0)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:35.755323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38616077-a11a-42d9-983c-868ec33a4430 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:35.757280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 542/1077] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:31:35 2022] GET /identity/v3/users/6013afe9b524409b83b13ea700b832a0 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:35.757928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38616077-a11a-42d9-983c-868ec33a4430 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:35.766543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f2311-bdfb-4737-9151-f4bf480dee1c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:35.800799 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285ceca-d631-49aa-94ab-66be7ca5edb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:35.801622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285ceca-d631-49aa-94ab-66be7ca5edb1 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.801906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285ceca-d631-49aa-94ab-66be7ca5edb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.802185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285ceca-d631-49aa-94ab-66be7ca5edb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.809157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285ceca-d631-49aa-94ab-66be7ca5edb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:35.811031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285ceca-d631-49aa-94ab-66be7ca5edb1 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d285ceca-d631-49aa-94ab-66be7ca5edb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:31:35.825238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:35.832460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:35.833611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:31:35.833611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:31:35.833611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:31:35.833611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:31:35.833611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:31:35.833611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:31:35.833611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:35.833611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:35.833611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 536/1078] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:35.833611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285ceca-d631-49aa-94ab-66be7ca5edb1 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:35.904718 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38616077-a11a-42d9-983c-868ec33a4430 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:35.942209 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ab1ec-e94f-4140-a8ce-0f2538062b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:35.943009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ab1ec-e94f-4140-a8ce-0f2538062b9f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:35.943294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ab1ec-e94f-4140-a8ce-0f2538062b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:35.943561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ab1ec-e94f-4140-a8ce-0f2538062b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:35.954670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ab1ec-e94f-4140-a8ce-0f2538062b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:35.956139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ab1ec-e94f-4140-a8ce-0f2538062b9f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:35.973176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285ceca-d631-49aa-94ab-66be7ca5edb1 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-361ab1ec-e94f-4140-a8ce-0f2538062b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:35.973753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:35.974770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:35.975726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:35.975726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:35.975726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:35.975726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:35.975726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 543/1079] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:35.976019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ab1ec-e94f-4140-a8ce-0f2538062b9f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:36.014179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc4e70-1269-4869-92ca-74ca7c339c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:36.015042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc4e70-1269-4869-92ca-74ca7c339c12 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:36.015331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc4e70-1269-4869-92ca-74ca7c339c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:36.015610 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc4e70-1269-4869-92ca-74ca7c339c12 None admin] PATH_INFO: `/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:36.016631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:36.016820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc4e70-1269-4869-92ca-74ca7c339c12 None admin] RBAC: Authorizing `identity:get_project(project_id=e3dfff089a14434d896cbd3f4ed0ea45)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:36.018337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc4e70-1269-4869-92ca-74ca7c339c12 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:36.020049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 537/1080] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:31:35 2022] GET /identity/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45 => generated 311 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:36.020821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc4e70-1269-4869-92ca-74ca7c339c12 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:36.023537 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:36.083213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ab1ec-e94f-4140-a8ce-0f2538062b9f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:36.118316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e868c4-8266-422b-bf6f-4c5e224150c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:36.118970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e868c4-8266-422b-bf6f-4c5e224150c7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:36.119251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e868c4-8266-422b-bf6f-4c5e224150c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:36.119524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e868c4-8266-422b-bf6f-4c5e224150c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:36.120311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e868c4-8266-422b-bf6f-4c5e224150c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:36.120656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e868c4-8266-422b-bf6f-4c5e224150c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:36.122148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e868c4-8266-422b-bf6f-4c5e224150c7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:36.123440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc4e70-1269-4869-92ca-74ca7c339c12 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:36.134739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 544/1081] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:31:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:36.135536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e868c4-8266-422b-bf6f-4c5e224150c7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:36.157794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead0c72-58ed-4773-8541-cf0ad27a61ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:36.158451 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead0c72-58ed-4773-8541-cf0ad27a61ff None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:36.158731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead0c72-58ed-4773-8541-cf0ad27a61ff None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:36.159009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead0c72-58ed-4773-8541-cf0ad27a61ff None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:36.159732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead0c72-58ed-4773-8541-cf0ad27a61ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:36.160036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead0c72-58ed-4773-8541-cf0ad27a61ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:36.161432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead0c72-58ed-4773-8541-cf0ad27a61ff None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:36.173916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 538/1082] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:36.174774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead0c72-58ed-4773-8541-cf0ad27a61ff None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:36.250566 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:36.265436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:36.270726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e868c4-8266-422b-bf6f-4c5e224150c7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:36.301370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead0c72-58ed-4773-8541-cf0ad27a61ff None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:36.310342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3a7b4-6d4b-4789-ba38-e3ce798ecce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:36.311366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3a7b4-6d4b-4789-ba38-e3ce798ecce1 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:36.311658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3a7b4-6d4b-4789-ba38-e3ce798ecce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:36.311939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3a7b4-6d4b-4789-ba38-e3ce798ecce1 None admin] PATH_INFO: `/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45/users/6013afe9b524409b83b13ea700b832a0/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:36.314223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3a7b4-6d4b-4789-ba38-e3ce798ecce1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3dfff089a14434d896cbd3f4ed0ea45, user_id=6013afe9b524409b83b13ea700b832a0, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:36.315702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3a7b4-6d4b-4789-ba38-e3ce798ecce1 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:36.343831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d37f4f-08e2-486f-8728-7792d4efd7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:36.344651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d37f4f-08e2-486f-8728-7792d4efd7dd None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:36.344974 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d37f4f-08e2-486f-8728-7792d4efd7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:36.345255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d37f4f-08e2-486f-8728-7792d4efd7dd None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/69ee145599384c6da2cba85a42832dcd/roles/443cff586282466c87ffce62f1440b63` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:36.347488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d37f4f-08e2-486f-8728-7792d4efd7dd None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=69ee145599384c6da2cba85a42832dcd, role_id=443cff586282466c87ffce62f1440b63)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:36.349015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d37f4f-08e2-486f-8728-7792d4efd7dd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:36.390665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 545/1083] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:31:36 2022] PUT /identity/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45/users/6013afe9b524409b83b13ea700b832a0/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:31:36.421185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 539/1084] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:31:36 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/69ee145599384c6da2cba85a42832dcd/roles/443cff586282466c87ffce62f1440b63 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:31:38.420963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ff3b8-e886-4223-b911-ed36c517dca0 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:38.421496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ff3b8-e886-4223-b911-ed36c517dca0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:38.421496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ff3b8-e886-4223-b911-ed36c517dca0 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:38.422876 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 546/1085] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:38.428342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efa97c-23b2-4395-a66c-b4c2a03c642a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:38.428764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efa97c-23b2-4395-a66c-b4c2a03c642a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:38.428929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efa97c-23b2-4395-a66c-b4c2a03c642a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:38.440547 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32961d4d-7b3f-4f9e-a514-702a20741c69 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:38.441417 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32961d4d-7b3f-4f9e-a514-702a20741c69 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:38.441606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32961d4d-7b3f-4f9e-a514-702a20741c69 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:38.442962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 547/1086] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:38.446909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-14efa97c-23b2-4395-a66c-b4c2a03c642a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:38.447901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-14efa97c-23b2-4395-a66c-b4c2a03c642a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:38.448016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab26ba-4acf-4b41-afe5-02901182b1e8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:38.448291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab26ba-4acf-4b41-afe5-02901182b1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:38.448565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab26ba-4acf-4b41-afe5-02901182b1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:38.464609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-22ab26ba-4acf-4b41-afe5-02901182b1e8 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:38.465889 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-22ab26ba-4acf-4b41-afe5-02901182b1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:38.516153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 540/1087] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:38.532700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 548/1088] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:38.539353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efa97c-23b2-4395-a66c-b4c2a03c642a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:38.555806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab26ba-4acf-4b41-afe5-02901182b1e8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:38.603953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:38.644696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:38.652377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efa97c-23b2-4395-a66c-b4c2a03c642a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:38.656496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab26ba-4acf-4b41-afe5-02901182b1e8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:38.691607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915f522-f724-4868-be53-1036cc3fafd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:38.691830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39685-3193-4ec5-a2cb-02781fda6749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:38.692318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39685-3193-4ec5-a2cb-02781fda6749 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:38.692424 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915f522-f724-4868-be53-1036cc3fafd8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:38.692597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915f522-f724-4868-be53-1036cc3fafd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:38.692700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39685-3193-4ec5-a2cb-02781fda6749 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:38.692870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915f522-f724-4868-be53-1036cc3fafd8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:38.693017 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39685-3193-4ec5-a2cb-02781fda6749 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:38.704605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b39685-3193-4ec5-a2cb-02781fda6749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:38.704896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915f522-f724-4868-be53-1036cc3fafd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b1b39685-3193-4ec5-a2cb-02781fda6749 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:38.712550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:38.713575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:31:38.715022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-4915f522-f724-4868-be53-1036cc3fafd8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:38.715667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:38.716512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 16:31:38.717434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:38.718054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 541/1089] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:31:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:38.718130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 549/1090] 10.222.0.88 () {56 vars in 1120 bytes} [Mon Jul 18 16:31:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:38.720784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915f522-f724-4868-be53-1036cc3fafd8 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:38.720927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39685-3193-4ec5-a2cb-02781fda6749 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:38.824561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915f522-f724-4868-be53-1036cc3fafd8 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:38.824778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39685-3193-4ec5-a2cb-02781fda6749 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:38.838011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:38.858938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c21cd-d2f1-4bd2-a78a-c909493a1dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:38.859159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b54f5-ec16-4563-9486-c1ed7164330b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:38.859611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c21cd-d2f1-4bd2-a78a-c909493a1dba None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:38.859707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b54f5-ec16-4563-9486-c1ed7164330b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:38.859891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c21cd-d2f1-4bd2-a78a-c909493a1dba None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:38.859980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b54f5-ec16-4563-9486-c1ed7164330b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:38.860162 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c21cd-d2f1-4bd2-a78a-c909493a1dba None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:38.860252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b54f5-ec16-4563-9486-c1ed7164330b None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:38.860930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b54f5-ec16-4563-9486-c1ed7164330b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:38.861035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c21cd-d2f1-4bd2-a78a-c909493a1dba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:38.861236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c21cd-d2f1-4bd2-a78a-c909493a1dba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:38.861333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b54f5-ec16-4563-9486-c1ed7164330b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:38.862607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c21cd-d2f1-4bd2-a78a-c909493a1dba None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:38.862799 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b54f5-ec16-4563-9486-c1ed7164330b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:38.871957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 550/1091] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Jul 18 16:31:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:38.872619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 542/1092] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:31:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:38.877635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c21cd-d2f1-4bd2-a78a-c909493a1dba None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:38.878053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b54f5-ec16-4563-9486-c1ed7164330b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:38.891228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:38.979447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c21cd-d2f1-4bd2-a78a-c909493a1dba None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:38.987352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b54f5-ec16-4563-9486-c1ed7164330b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:39.014463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395274e-ab46-45fe-8983-e93438dd7a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:39.015256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395274e-ab46-45fe-8983-e93438dd7a47 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:39.015537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395274e-ab46-45fe-8983-e93438dd7a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:39.016007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395274e-ab46-45fe-8983-e93438dd7a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:39.023475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b7839-76bc-4108-9d14-40f5ea96aff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:39.024356 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7839-76bc-4108-9d14-40f5ea96aff7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:39.024638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7839-76bc-4108-9d14-40f5ea96aff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:39.024943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7839-76bc-4108-9d14-40f5ea96aff7 None admin] PATH_INFO: `/v3/users/6013afe9b524409b83b13ea700b832a0` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:39.026179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7839-76bc-4108-9d14-40f5ea96aff7 None admin] RBAC: Authorizing `identity:get_user(user_id=6013afe9b524409b83b13ea700b832a0)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:39.026864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395274e-ab46-45fe-8983-e93438dd7a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:39.027680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7839-76bc-4108-9d14-40f5ea96aff7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:39.028287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395274e-ab46-45fe-8983-e93438dd7a47 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:39.029363 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 543/1093] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:31:38 2022] GET /identity/v3/users/6013afe9b524409b83b13ea700b832a0 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:39.034942 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b7839-76bc-4108-9d14-40f5ea96aff7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-3395274e-ab46-45fe-8983-e93438dd7a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:39.049224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:39.050167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:39.051061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:39.051061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:39.051061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:39.051061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:39.051061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 551/1094] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:39.056075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395274e-ab46-45fe-8983-e93438dd7a47 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:39.080623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:39.118865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:39.143194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b7839-76bc-4108-9d14-40f5ea96aff7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:39.158772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395274e-ab46-45fe-8983-e93438dd7a47 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:39.177787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97028ba8-569c-40ca-9cf2-8b0c52bb2d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:39.178457 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97028ba8-569c-40ca-9cf2-8b0c52bb2d75 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:39.178742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97028ba8-569c-40ca-9cf2-8b0c52bb2d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:39.179016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97028ba8-569c-40ca-9cf2-8b0c52bb2d75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:39.192615 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354750c9-ae77-418b-aac3-ccf47286591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:39.193313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354750c9-ae77-418b-aac3-ccf47286591e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:39.193601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354750c9-ae77-418b-aac3-ccf47286591e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:39.193880 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354750c9-ae77-418b-aac3-ccf47286591e None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:39.194608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354750c9-ae77-418b-aac3-ccf47286591e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:39.194915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354750c9-ae77-418b-aac3-ccf47286591e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:39.196293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354750c9-ae77-418b-aac3-ccf47286591e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:39.208750 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 552/1095] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:39.215047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354750c9-ae77-418b-aac3-ccf47286591e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:39.253021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97028ba8-569c-40ca-9cf2-8b0c52bb2d75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:39.254496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97028ba8-569c-40ca-9cf2-8b0c52bb2d75 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:39.298978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:39.325994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354750c9-ae77-418b-aac3-ccf47286591e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:39.349226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:39.357457 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 544/1096] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:31:39.361329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffc92a-a24f-4b73-a215-e9aa79bd3135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:39.362441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffc92a-a24f-4b73-a215-e9aa79bd3135 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:39.362733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffc92a-a24f-4b73-a215-e9aa79bd3135 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:39.363011 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffc92a-a24f-4b73-a215-e9aa79bd3135 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:39.363950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97028ba8-569c-40ca-9cf2-8b0c52bb2d75 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:39.372276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffc92a-a24f-4b73-a215-e9aa79bd3135 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:39.374386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beffc92a-a24f-4b73-a215-e9aa79bd3135 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-beffc92a-a24f-4b73-a215-e9aa79bd3135 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:31:39.389514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:39.390468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:39.391406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:39.391406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:39.391406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:31:39.391406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:31:39.391406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:31:39.391406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:31:39.391406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:31:39.391406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:31:39.391406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 16:31:39.391406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:39.391406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 553/1097] 10.222.0.88 () {56 vars in 1093 bytes} [Mon Jul 18 16:31:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:31:39.396537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffc92a-a24f-4b73-a215-e9aa79bd3135 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:39.463068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97028ba8-569c-40ca-9cf2-8b0c52bb2d75 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:39.497159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffc92a-a24f-4b73-a215-e9aa79bd3135 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:39.497799 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c0c0b-d93c-47eb-8ba0-9e3392602129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:39.498662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c0c0b-d93c-47eb-8ba0-9e3392602129 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:39.498939 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c0c0b-d93c-47eb-8ba0-9e3392602129 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:39.499215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c0c0b-d93c-47eb-8ba0-9e3392602129 None admin] PATH_INFO: `/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:39.500381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c0c0b-d93c-47eb-8ba0-9e3392602129 None admin] RBAC: Authorizing `identity:get_project(project_id=e3dfff089a14434d896cbd3f4ed0ea45)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:39.501855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c0c0b-d93c-47eb-8ba0-9e3392602129 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:39.503595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 545/1098] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:31:39 2022] GET /identity/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:39.509492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c0c0b-d93c-47eb-8ba0-9e3392602129 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:39.522201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:39.533322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860a0a6-2850-4011-9870-7ad88e7881f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:39.533995 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860a0a6-2850-4011-9870-7ad88e7881f6 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:39.534282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860a0a6-2850-4011-9870-7ad88e7881f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:39.534563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860a0a6-2850-4011-9870-7ad88e7881f6 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:39.535395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860a0a6-2850-4011-9870-7ad88e7881f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:39.535783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860a0a6-2850-4011-9870-7ad88e7881f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:39.537255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860a0a6-2850-4011-9870-7ad88e7881f6 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:39.554940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 554/1099] 10.222.0.88 () {56 vars in 1133 bytes} [Mon Jul 18 16:31:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:39.561195 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860a0a6-2850-4011-9870-7ad88e7881f6 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:39.608222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:39.614517 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c0c0b-d93c-47eb-8ba0-9e3392602129 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:39.650896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f0ab5-710b-4b46-987b-1a51ef95f6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:39.651560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f0ab5-710b-4b46-987b-1a51ef95f6ea None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:39.651847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f0ab5-710b-4b46-987b-1a51ef95f6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:39.652126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f0ab5-710b-4b46-987b-1a51ef95f6ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:39.652988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f0ab5-710b-4b46-987b-1a51ef95f6ea None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=e3dfff089a14434d896cbd3f4ed0ea45, user.id=6013afe9b524409b83b13ea700b832a0) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:39.653298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f0ab5-710b-4b46-987b-1a51ef95f6ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:39.654735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f0ab5-710b-4b46-987b-1a51ef95f6ea None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:39.676915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 546/1100] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:39 2022] GET /identity/v3/role_assignments?user.id=6013afe9b524409b83b13ea700b832a0&scope.project.id=e3dfff089a14434d896cbd3f4ed0ea45&role.id=81457691f3a2456fa33358cb6a922d05 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:31:39.677143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860a0a6-2850-4011-9870-7ad88e7881f6 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:39.709606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf5b0a-75df-48ed-8846-6410a8e99872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:39.710272 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf5b0a-75df-48ed-8846-6410a8e99872 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:39.710553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf5b0a-75df-48ed-8846-6410a8e99872 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:39.710827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf5b0a-75df-48ed-8846-6410a8e99872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:39.786964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf5b0a-75df-48ed-8846-6410a8e99872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:39.788372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf5b0a-75df-48ed-8846-6410a8e99872 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:39.833001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:39.894382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 555/1101] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:31:39.900290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f0ab5-710b-4b46-987b-1a51ef95f6ea None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:39.977733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:40.003144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f0ab5-710b-4b46-987b-1a51ef95f6ea None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:40.039017 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ef3a7-0c46-4aa2-b2c4-719352424777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:40.039727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ef3a7-0c46-4aa2-b2c4-719352424777 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:40.040001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ef3a7-0c46-4aa2-b2c4-719352424777 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:40.040272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ef3a7-0c46-4aa2-b2c4-719352424777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:40.051012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ef3a7-0c46-4aa2-b2c4-719352424777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:40.052450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ef3a7-0c46-4aa2-b2c4-719352424777 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-6c6ef3a7-0c46-4aa2-b2c4-719352424777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:31:40.070294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:40.071358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:40.072259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:31:40.072259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:31:40.072259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:31:40.072259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:40.072259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 547/1102] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:40.077154 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf5b0a-75df-48ed-8846-6410a8e99872 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:40.178525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf5b0a-75df-48ed-8846-6410a8e99872 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:40.212302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca75a38-2422-446f-b179-2c67bcfd600b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:40.213003 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca75a38-2422-446f-b179-2c67bcfd600b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:40.213292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca75a38-2422-446f-b179-2c67bcfd600b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:40.213571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca75a38-2422-446f-b179-2c67bcfd600b None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:40.214833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca75a38-2422-446f-b179-2c67bcfd600b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:40.215145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca75a38-2422-446f-b179-2c67bcfd600b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:40.216531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca75a38-2422-446f-b179-2c67bcfd600b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:40.225139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:40.228682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 556/1103] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:40.234862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ef3a7-0c46-4aa2-b2c4-719352424777 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:40.335845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ef3a7-0c46-4aa2-b2c4-719352424777 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:40.364163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:40.373466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15e6eb-371d-48e5-8d18-5e18b875a682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:40.374268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15e6eb-371d-48e5-8d18-5e18b875a682 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:40.374554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15e6eb-371d-48e5-8d18-5e18b875a682 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:40.374834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15e6eb-371d-48e5-8d18-5e18b875a682 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:40.382450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15e6eb-371d-48e5-8d18-5e18b875a682 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:40.383917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15e6eb-371d-48e5-8d18-5e18b875a682 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-eb15e6eb-371d-48e5-8d18-5e18b875a682 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:31:40.398728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:40.399713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:40.400640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:31:40.400640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:31:40.400640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:31:40.400640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:31:40.400640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:31:40.400640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:31:40.400640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:31:40.400640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:31:40.400640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 548/1104] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:31:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:40.406534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca75a38-2422-446f-b179-2c67bcfd600b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:40.509806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca75a38-2422-446f-b179-2c67bcfd600b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:40.543802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a559bb59-9a9a-4d9c-a314-c150bdb65c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:40.544460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559bb59-9a9a-4d9c-a314-c150bdb65c56 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:40.544747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559bb59-9a9a-4d9c-a314-c150bdb65c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:40.545071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a559bb59-9a9a-4d9c-a314-c150bdb65c56 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:40.545863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559bb59-9a9a-4d9c-a314-c150bdb65c56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:40.546173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559bb59-9a9a-4d9c-a314-c150bdb65c56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:40.547649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a559bb59-9a9a-4d9c-a314-c150bdb65c56 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:40.559743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 557/1105] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:31:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:40.566033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15e6eb-371d-48e5-8d18-5e18b875a682 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:40.677870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15e6eb-371d-48e5-8d18-5e18b875a682 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:40.712401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636be3d-1661-4255-8898-e7518a658dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:40.713081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636be3d-1661-4255-8898-e7518a658dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:40.713364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636be3d-1661-4255-8898-e7518a658dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:40.713637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636be3d-1661-4255-8898-e7518a658dc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:40.715014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636be3d-1661-4255-8898-e7518a658dc8 None admin] RBAC: Adding query filter params (role.id=443cff586282466c87ffce62f1440b63, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=69ee145599384c6da2cba85a42832dcd) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:40.715320 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636be3d-1661-4255-8898-e7518a658dc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:40.716749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636be3d-1661-4255-8898-e7518a658dc8 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:40.723194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 549/1106] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:40 2022] GET /identity/v3/role_assignments?user.id=69ee145599384c6da2cba85a42832dcd&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=443cff586282466c87ffce62f1440b63 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:41.710743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2cb4f-eaf2-4021-95d4-d0ae0caf2bee None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:41.711299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2cb4f-eaf2-4021-95d4-d0ae0caf2bee None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:41.711299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2cb4f-eaf2-4021-95d4-d0ae0caf2bee None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:41.712670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 558/1107] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:41.718345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fdbe8-fd0e-4412-81b6-87cf8b85d673 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:41.718790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fdbe8-fd0e-4412-81b6-87cf8b85d673 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:41.718894 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fdbe8-fd0e-4412-81b6-87cf8b85d673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:41.735851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7f2fdbe8-fd0e-4412-81b6-87cf8b85d673 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:41.737062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7f2fdbe8-fd0e-4412-81b6-87cf8b85d673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:41.781864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:41.802148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 550/1108] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:41.826082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2cb4f-eaf2-4021-95d4-d0ae0caf2bee None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:41.886776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:41.928097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2cb4f-eaf2-4021-95d4-d0ae0caf2bee None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:41.961928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599708f6-a1df-443d-899f-90305c61d584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:41.962641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599708f6-a1df-443d-899f-90305c61d584 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:41.962938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599708f6-a1df-443d-899f-90305c61d584 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:41.963209 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599708f6-a1df-443d-899f-90305c61d584 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:41.965029 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599708f6-a1df-443d-899f-90305c61d584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:41.966459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599708f6-a1df-443d-899f-90305c61d584 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:41.968074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 559/1109] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:41.974181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fdbe8-fd0e-4412-81b6-87cf8b85d673 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:42.071183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fdbe8-fd0e-4412-81b6-87cf8b85d673 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:42.104961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a44ce8-6020-445d-ab0e-9270a7b5289a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:42.105627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a44ce8-6020-445d-ab0e-9270a7b5289a None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:42.105914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a44ce8-6020-445d-ab0e-9270a7b5289a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:42.106193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a44ce8-6020-445d-ab0e-9270a7b5289a None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:42.106905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a44ce8-6020-445d-ab0e-9270a7b5289a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:42.108334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a44ce8-6020-445d-ab0e-9270a7b5289a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:42.219255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 551/1110] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:31:41 2022] POST /identity/v3/users => generated 286 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:44.257048 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4ccbc-a7a9-40f2-88ec-616b939792a6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:44.257588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4ccbc-a7a9-40f2-88ec-616b939792a6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:44.257588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4ccbc-a7a9-40f2-88ec-616b939792a6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:44.258971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 560/1111] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:44.265205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85028943-958d-4ba3-ae6d-092a1c9dec54 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:44.265650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85028943-958d-4ba3-ae6d-092a1c9dec54 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:44.265768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85028943-958d-4ba3-ae6d-092a1c9dec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:44.283841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-85028943-958d-4ba3-ae6d-092a1c9dec54 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:44.285329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-85028943-958d-4ba3-ae6d-092a1c9dec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:44.315232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:44.352642 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 552/1112] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:44.377569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4ccbc-a7a9-40f2-88ec-616b939792a6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:44.478815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4ccbc-a7a9-40f2-88ec-616b939792a6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:44.505646 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:44.513344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f851f77-121a-41cd-9113-cac70c487c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:44.514178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f851f77-121a-41cd-9113-cac70c487c60 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:44.514461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f851f77-121a-41cd-9113-cac70c487c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:44.514733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f851f77-121a-41cd-9113-cac70c487c60 None admin] PATH_INFO: `/v3/roles/c465d586f0ec4b59815b23087d757fd2` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:44.516202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f851f77-121a-41cd-9113-cac70c487c60 None admin] RBAC: Authorizing `identity:get_role(role_id=c465d586f0ec4b59815b23087d757fd2)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:44.517605 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f851f77-121a-41cd-9113-cac70c487c60 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:44.518970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 561/1113] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:31:44 2022] GET /identity/v3/roles/c465d586f0ec4b59815b23087d757fd2 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:44.524985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85028943-958d-4ba3-ae6d-092a1c9dec54 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:44.625298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85028943-958d-4ba3-ae6d-092a1c9dec54 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:44.659722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30636efe-92f3-4247-871b-e82822aa14ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:44.660595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30636efe-92f3-4247-871b-e82822aa14ea None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:44.660897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30636efe-92f3-4247-871b-e82822aa14ea None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:44.661187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30636efe-92f3-4247-871b-e82822aa14ea None admin] PATH_INFO: `/v3/users/003b1ab389de47f584d23e41daf2d624` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:44.675081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30636efe-92f3-4247-871b-e82822aa14ea None admin] RBAC: Authorizing `identity:get_user(user_id=003b1ab389de47f584d23e41daf2d624)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:44.676520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30636efe-92f3-4247-871b-e82822aa14ea None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:44.678216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 553/1114] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:31:44 2022] GET /identity/v3/users/003b1ab389de47f584d23e41daf2d624 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:44.684236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f851f77-121a-41cd-9113-cac70c487c60 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:44.786932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f851f77-121a-41cd-9113-cac70c487c60 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:44.821250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54130f81-459c-4901-8381-eacf66d85b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:44.821893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54130f81-459c-4901-8381-eacf66d85b97 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:44.822175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54130f81-459c-4901-8381-eacf66d85b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:44.822448 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54130f81-459c-4901-8381-eacf66d85b97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:44.883383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:44.889883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54130f81-459c-4901-8381-eacf66d85b97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:44.891380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54130f81-459c-4901-8381-eacf66d85b97 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:45.044209 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 562/1115] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:31:45.050140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30636efe-92f3-4247-871b-e82822aa14ea None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:45.075452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:45.149169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30636efe-92f3-4247-871b-e82822aa14ea None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:45.182817 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3812ae3-4327-4643-927e-827059d2e65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:45.183710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3812ae3-4327-4643-927e-827059d2e65a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:45.183984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3812ae3-4327-4643-927e-827059d2e65a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:45.184254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3812ae3-4327-4643-927e-827059d2e65a None admin] PATH_INFO: `/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:45.185464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3812ae3-4327-4643-927e-827059d2e65a None admin] RBAC: Authorizing `identity:get_project(project_id=e3dfff089a14434d896cbd3f4ed0ea45)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:45.186888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3812ae3-4327-4643-927e-827059d2e65a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:45.188651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 554/1116] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:31:45 2022] GET /identity/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:45.194921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54130f81-459c-4901-8381-eacf66d85b97 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:45.290320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:45.296174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54130f81-459c-4901-8381-eacf66d85b97 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:45.329833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3964b26-0121-4a5c-a4fa-cd4cee8b4ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:45.330477 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964b26-0121-4a5c-a4fa-cd4cee8b4ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:45.330760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964b26-0121-4a5c-a4fa-cd4cee8b4ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:45.331035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3964b26-0121-4a5c-a4fa-cd4cee8b4ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:45.331900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3964b26-0121-4a5c-a4fa-cd4cee8b4ab8 None admin] RBAC: Adding query filter params (role.id=c465d586f0ec4b59815b23087d757fd2, scope.project.id=e3dfff089a14434d896cbd3f4ed0ea45, user.id=003b1ab389de47f584d23e41daf2d624) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:45.332210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3964b26-0121-4a5c-a4fa-cd4cee8b4ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:45.333674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3964b26-0121-4a5c-a4fa-cd4cee8b4ab8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:45.339658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 563/1117] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:45 2022] GET /identity/v3/role_assignments?user.id=003b1ab389de47f584d23e41daf2d624&scope.project.id=e3dfff089a14434d896cbd3f4ed0ea45&role.id=c465d586f0ec4b59815b23087d757fd2 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:47.296671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6ca51-65df-4259-9072-e1578e846680 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:47.297293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6ca51-65df-4259-9072-e1578e846680 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:47.297293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6ca51-65df-4259-9072-e1578e846680 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:47.298661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 555/1118] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:47.303882 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57842a-3761-4422-bb59-a2133d8ede0f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:47.304313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57842a-3761-4422-bb59-a2133d8ede0f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:47.304447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57842a-3761-4422-bb59-a2133d8ede0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:47.321708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5e57842a-3761-4422-bb59-a2133d8ede0f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:47.322907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5e57842a-3761-4422-bb59-a2133d8ede0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:47.390967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 564/1119] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:47.413339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6ca51-65df-4259-9072-e1578e846680 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:47.506243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:47.515714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6ca51-65df-4259-9072-e1578e846680 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:47.549602 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4b86f-fd5a-4726-8523-84e00a40c993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:47.550378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4b86f-fd5a-4726-8523-84e00a40c993 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:47.550659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4b86f-fd5a-4726-8523-84e00a40c993 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:47.550936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4b86f-fd5a-4726-8523-84e00a40c993 None admin] PATH_INFO: `/v3/roles/c465d586f0ec4b59815b23087d757fd2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:47.552391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4b86f-fd5a-4726-8523-84e00a40c993 None admin] RBAC: Authorizing `identity:get_role(role_id=c465d586f0ec4b59815b23087d757fd2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:47.553776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4b86f-fd5a-4726-8523-84e00a40c993 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:47.555144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 556/1120] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:31:47 2022] GET /identity/v3/roles/c465d586f0ec4b59815b23087d757fd2 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:47.561735 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57842a-3761-4422-bb59-a2133d8ede0f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:47.649474 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:47.659168 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57842a-3761-4422-bb59-a2133d8ede0f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:47.694300 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4acbf-6963-4935-9f63-9555367b82d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:47.695178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4acbf-6963-4935-9f63-9555367b82d5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:47.695462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4acbf-6963-4935-9f63-9555367b82d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:47.695738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4acbf-6963-4935-9f63-9555367b82d5 None admin] PATH_INFO: `/v3/users/003b1ab389de47f584d23e41daf2d624` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:47.696970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4acbf-6963-4935-9f63-9555367b82d5 None admin] RBAC: Authorizing `identity:get_user(user_id=003b1ab389de47f584d23e41daf2d624)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:47.698455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4acbf-6963-4935-9f63-9555367b82d5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:47.700117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 565/1121] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:31:47 2022] GET /identity/v3/users/003b1ab389de47f584d23e41daf2d624 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:47.706059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4b86f-fd5a-4726-8523-84e00a40c993 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:47.804633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4b86f-fd5a-4726-8523-84e00a40c993 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:47.837365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7689ae-3557-4db9-9840-b0d00b9b088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:47.838216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7689ae-3557-4db9-9840-b0d00b9b088b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:47.838498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7689ae-3557-4db9-9840-b0d00b9b088b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:47.839276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7689ae-3557-4db9-9840-b0d00b9b088b None admin] PATH_INFO: `/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:47.840516 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7689ae-3557-4db9-9840-b0d00b9b088b None admin] RBAC: Authorizing `identity:get_project(project_id=e3dfff089a14434d896cbd3f4ed0ea45)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:47.841979 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7689ae-3557-4db9-9840-b0d00b9b088b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:47.843665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 557/1122] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:31:47 2022] GET /identity/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:47.849808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4acbf-6963-4935-9f63-9555367b82d5 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:47.952085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4acbf-6963-4935-9f63-9555367b82d5 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:47.986511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc8d96-348d-4a79-a438-36e4e4d6b69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:47.987419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc8d96-348d-4a79-a438-36e4e4d6b69a None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:47.987692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc8d96-348d-4a79-a438-36e4e4d6b69a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:47.987960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc8d96-348d-4a79-a438-36e4e4d6b69a None admin] PATH_INFO: `/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45/users/003b1ab389de47f584d23e41daf2d624/roles/c465d586f0ec4b59815b23087d757fd2` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:47.990163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc8d96-348d-4a79-a438-36e4e4d6b69a None admin] RBAC: Authorizing `identity:create_grant(project_id=e3dfff089a14434d896cbd3f4ed0ea45, user_id=003b1ab389de47f584d23e41daf2d624, role_id=c465d586f0ec4b59815b23087d757fd2)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:47.991849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc8d96-348d-4a79-a438-36e4e4d6b69a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:48.076286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 566/1123] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:31:47 2022] PUT /identity/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45/users/003b1ab389de47f584d23e41daf2d624/roles/c465d586f0ec4b59815b23087d757fd2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:31:50.056409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3f83-5ca8-4fc4-bcc0-fa47254696cb None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:50.056983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3f83-5ca8-4fc4-bcc0-fa47254696cb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:50.056983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c3f83-5ca8-4fc4-bcc0-fa47254696cb None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:50.058945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 558/1124] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:50.064216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f459-0039-45d3-ad52-988d670151bd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:50.064654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f459-0039-45d3-ad52-988d670151bd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:50.064778 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90f459-0039-45d3-ad52-988d670151bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:50.081865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fb90f459-0039-45d3-ad52-988d670151bd None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:50.083064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fb90f459-0039-45d3-ad52-988d670151bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:50.096189 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:50.149668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 567/1125] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:50.172483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c3f83-5ca8-4fc4-bcc0-fa47254696cb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:50.201251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:50.274552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c3f83-5ca8-4fc4-bcc0-fa47254696cb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:50.308775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e8b72-6285-43a1-8ea0-147784a09d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:50.309653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e8b72-6285-43a1-8ea0-147784a09d3c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:50.309937 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e8b72-6285-43a1-8ea0-147784a09d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:50.310211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e8b72-6285-43a1-8ea0-147784a09d3c None admin] PATH_INFO: `/v3/roles/c465d586f0ec4b59815b23087d757fd2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:50.311685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e8b72-6285-43a1-8ea0-147784a09d3c None admin] RBAC: Authorizing `identity:get_role(role_id=c465d586f0ec4b59815b23087d757fd2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:50.313084 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e8b72-6285-43a1-8ea0-147784a09d3c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:50.314456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 559/1126] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:31:50 2022] GET /identity/v3/roles/c465d586f0ec4b59815b23087d757fd2 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:50.320459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90f459-0039-45d3-ad52-988d670151bd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:50.421096 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90f459-0039-45d3-ad52-988d670151bd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:50.455576 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467b790-c396-470d-8f17-908cd9a37288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:50.456374 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467b790-c396-470d-8f17-908cd9a37288 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:50.456655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467b790-c396-470d-8f17-908cd9a37288 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:50.456956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467b790-c396-470d-8f17-908cd9a37288 None admin] PATH_INFO: `/v3/users/003b1ab389de47f584d23e41daf2d624` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:50.458155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467b790-c396-470d-8f17-908cd9a37288 None admin] RBAC: Authorizing `identity:get_user(user_id=003b1ab389de47f584d23e41daf2d624)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:50.459603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467b790-c396-470d-8f17-908cd9a37288 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:50.461290 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 568/1127] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:31:50 2022] GET /identity/v3/users/003b1ab389de47f584d23e41daf2d624 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:50.467904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e8b72-6285-43a1-8ea0-147784a09d3c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:50.561007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:50.566825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e8b72-6285-43a1-8ea0-147784a09d3c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:50.601960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfaa3a9-7c0d-4c4c-bcbd-84bab41df055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:50.602614 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfaa3a9-7c0d-4c4c-bcbd-84bab41df055 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:50.602892 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfaa3a9-7c0d-4c4c-bcbd-84bab41df055 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:50.603161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfaa3a9-7c0d-4c4c-bcbd-84bab41df055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:50.669554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfaa3a9-7c0d-4c4c-bcbd-84bab41df055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:50.671017 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfaa3a9-7c0d-4c4c-bcbd-84bab41df055 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:50.763662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:50.770157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 560/1128] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:31:50.776078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467b790-c396-470d-8f17-908cd9a37288 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:50.793807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:50.876652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467b790-c396-470d-8f17-908cd9a37288 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:50.910095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4cebc-a17d-4675-be61-91f54a31f64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:50.910939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4cebc-a17d-4675-be61-91f54a31f64f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:50.911217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4cebc-a17d-4675-be61-91f54a31f64f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:50.911487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4cebc-a17d-4675-be61-91f54a31f64f None admin] PATH_INFO: `/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:50.912649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4cebc-a17d-4675-be61-91f54a31f64f None admin] RBAC: Authorizing `identity:get_project(project_id=e3dfff089a14434d896cbd3f4ed0ea45)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:50.914105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4cebc-a17d-4675-be61-91f54a31f64f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:50.915785 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 569/1129] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:31:50 2022] GET /identity/v3/projects/e3dfff089a14434d896cbd3f4ed0ea45 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:50.922724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfaa3a9-7c0d-4c4c-bcbd-84bab41df055 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:51.019993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfaa3a9-7c0d-4c4c-bcbd-84bab41df055 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:51.053402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6a303-3315-47e5-9ca7-423def8c8514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:51.054052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6a303-3315-47e5-9ca7-423def8c8514 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:51.054341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6a303-3315-47e5-9ca7-423def8c8514 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:51.054620 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6a303-3315-47e5-9ca7-423def8c8514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:51.055447 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6a303-3315-47e5-9ca7-423def8c8514 None admin] RBAC: Adding query filter params (role.id=c465d586f0ec4b59815b23087d757fd2, scope.project.id=e3dfff089a14434d896cbd3f4ed0ea45, user.id=003b1ab389de47f584d23e41daf2d624) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:51.055757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6a303-3315-47e5-9ca7-423def8c8514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:51.057208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6a303-3315-47e5-9ca7-423def8c8514 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:51.063066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 561/1130] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:50 2022] GET /identity/v3/role_assignments?user.id=003b1ab389de47f584d23e41daf2d624&scope.project.id=e3dfff089a14434d896cbd3f4ed0ea45&role.id=c465d586f0ec4b59815b23087d757fd2 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:53.086601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b590-68b6-4f73-96d3-56c9110c4f09 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:53.087156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b590-68b6-4f73-96d3-56c9110c4f09 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:53.087156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b590-68b6-4f73-96d3-56c9110c4f09 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:53.088518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 570/1131] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:31:53.093756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb168c-e1f3-44b6-981d-8c1c8d3967bf None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:53.094152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb168c-e1f3-44b6-981d-8c1c8d3967bf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:53.094319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb168c-e1f3-44b6-981d-8c1c8d3967bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:53.110727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7fbb168c-e1f3-44b6-981d-8c1c8d3967bf None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:53.111903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7fbb168c-e1f3-44b6-981d-8c1c8d3967bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:53.166901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:53.178021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 562/1132] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:53.202291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02b590-68b6-4f73-96d3-56c9110c4f09 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:53.286130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:53.303105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02b590-68b6-4f73-96d3-56c9110c4f09 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:53.337895 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95e883-560d-4490-8383-422d3ba72f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:53.338705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95e883-560d-4490-8383-422d3ba72f5f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:53.339016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95e883-560d-4490-8383-422d3ba72f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:53.339299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95e883-560d-4490-8383-422d3ba72f5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:53.340484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95e883-560d-4490-8383-422d3ba72f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:53.341953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95e883-560d-4490-8383-422d3ba72f5f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:53.343558 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 571/1133] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:53.349501 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb168c-e1f3-44b6-981d-8c1c8d3967bf None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:53.447212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb168c-e1f3-44b6-981d-8c1c8d3967bf None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:53.480503 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c017b-c341-42bc-9f84-09e90ae52d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:53.481118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c017b-c341-42bc-9f84-09e90ae52d5a None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:53.481408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c017b-c341-42bc-9f84-09e90ae52d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:53.481684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c017b-c341-42bc-9f84-09e90ae52d5a None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:53.482399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c017b-c341-42bc-9f84-09e90ae52d5a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:53.483830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c017b-c341-42bc-9f84-09e90ae52d5a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:53.607194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 563/1134] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:31:53 2022] POST /identity/v3/projects => generated 311 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:55.661108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897718b8-eabf-41f9-b059-2b046455d731 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:55.661665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897718b8-eabf-41f9-b059-2b046455d731 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:55.661665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897718b8-eabf-41f9-b059-2b046455d731 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:55.663036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 572/1135] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:55.668350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe312-cff3-49ce-b10b-01ff6905d00c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:55.668768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe312-cff3-49ce-b10b-01ff6905d00c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:55.668932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebe312-cff3-49ce-b10b-01ff6905d00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:55.686159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-39ebe312-cff3-49ce-b10b-01ff6905d00c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:55.687333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-39ebe312-cff3-49ce-b10b-01ff6905d00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:55.722093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:55.752187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 564/1136] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:31:55.777093 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897718b8-eabf-41f9-b059-2b046455d731 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:55.876943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897718b8-eabf-41f9-b059-2b046455d731 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:55.910941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad602ce9-511c-40cf-b426-b8f26de8397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:55.911783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad602ce9-511c-40cf-b426-b8f26de8397f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:55.912022 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad602ce9-511c-40cf-b426-b8f26de8397f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:55.912298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad602ce9-511c-40cf-b426-b8f26de8397f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:55.913783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad602ce9-511c-40cf-b426-b8f26de8397f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:55.915219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad602ce9-511c-40cf-b426-b8f26de8397f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:55.916849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 573/1137] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:31:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:55.922807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebe312-cff3-49ce-b10b-01ff6905d00c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:56.023599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebe312-cff3-49ce-b10b-01ff6905d00c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:56.056816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf7811-062d-4682-9350-36132e4ed497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:56.058050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf7811-062d-4682-9350-36132e4ed497 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:56.058333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf7811-062d-4682-9350-36132e4ed497 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:56.058607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf7811-062d-4682-9350-36132e4ed497 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:56.059308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf7811-062d-4682-9350-36132e4ed497 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:56.060736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf7811-062d-4682-9350-36132e4ed497 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:56.185104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 565/1138] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:31:55 2022] POST /identity/v3/users => generated 286 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:31:58.242636 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0cc48-eeea-4a0d-a611-924ebcea2ac1 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:58.243137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0cc48-eeea-4a0d-a611-924ebcea2ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:58.243137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0cc48-eeea-4a0d-a611-924ebcea2ac1 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:58.244475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 574/1139] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:31:58.249700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34246f06-e262-487e-9425-b8be88826196 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:58.250082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34246f06-e262-487e-9425-b8be88826196 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:58.250264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34246f06-e262-487e-9425-b8be88826196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:58.266854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-34246f06-e262-487e-9425-b8be88826196 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:31:58.268047 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-34246f06-e262-487e-9425-b8be88826196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:31:58.290951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:58.334707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 566/1140] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:31:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:31:58.359596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0cc48-eeea-4a0d-a611-924ebcea2ac1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:58.383671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:58.466129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0cc48-eeea-4a0d-a611-924ebcea2ac1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:58.499598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e992afb-578b-494b-8b73-818186a33231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:58.500331 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e992afb-578b-494b-8b73-818186a33231 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:58.500611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e992afb-578b-494b-8b73-818186a33231 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:58.500913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e992afb-578b-494b-8b73-818186a33231 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:58.511308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e992afb-578b-494b-8b73-818186a33231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6e992afb-578b-494b-8b73-818186a33231 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:31:58.518852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:31:58.519835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:31:58.520702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 575/1141] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:31:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:31:58.526856 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34246f06-e262-487e-9425-b8be88826196 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:58.624096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34246f06-e262-487e-9425-b8be88826196 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:58.659301 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4b5d2-833f-4ba0-bd11-80b6749e9e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:58.659967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4b5d2-833f-4ba0-bd11-80b6749e9e06 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:58.660249 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4b5d2-833f-4ba0-bd11-80b6749e9e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:58.660731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4b5d2-833f-4ba0-bd11-80b6749e9e06 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:58.661810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4b5d2-833f-4ba0-bd11-80b6749e9e06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:58.662092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4b5d2-833f-4ba0-bd11-80b6749e9e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:58.663466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4b5d2-833f-4ba0-bd11-80b6749e9e06 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:58.672668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 567/1142] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:31:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:58.679097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e992afb-578b-494b-8b73-818186a33231 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:58.775402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:58.794451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e992afb-578b-494b-8b73-818186a33231 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:58.828263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20587c0c-fa06-4653-8f02-339a4b6f793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:58.829165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20587c0c-fa06-4653-8f02-339a4b6f793e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:58.829449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20587c0c-fa06-4653-8f02-339a4b6f793e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:58.829723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20587c0c-fa06-4653-8f02-339a4b6f793e None admin] PATH_INFO: `/v3/users/806a72710a744fd1a7e9527fbe4f0d8b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:58.883905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20587c0c-fa06-4653-8f02-339a4b6f793e None admin] RBAC: Authorizing `identity:get_user(user_id=806a72710a744fd1a7e9527fbe4f0d8b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:58.885504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20587c0c-fa06-4653-8f02-339a4b6f793e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:58.887726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 576/1143] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:31:58 2022] GET /identity/v3/users/806a72710a744fd1a7e9527fbe4f0d8b => generated 286 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:58.893633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4b5d2-833f-4ba0-bd11-80b6749e9e06 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:58.916781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:58.998340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4b5d2-833f-4ba0-bd11-80b6749e9e06 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:59.032645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a04ef-57ea-4122-8261-2a53cef2d1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:59.033250 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a04ef-57ea-4122-8261-2a53cef2d1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:59.033539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a04ef-57ea-4122-8261-2a53cef2d1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:59.033820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a04ef-57ea-4122-8261-2a53cef2d1e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:59.107767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a04ef-57ea-4122-8261-2a53cef2d1e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:59.109389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a04ef-57ea-4122-8261-2a53cef2d1e9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:59.132251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:59.209626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 568/1144] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:31:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:31:59.215537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20587c0c-fa06-4653-8f02-339a4b6f793e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:59.314575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20587c0c-fa06-4653-8f02-339a4b6f793e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:59.347353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b78c97-6cd5-40dd-8c6a-e506e6602695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:59.348198 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b78c97-6cd5-40dd-8c6a-e506e6602695 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:59.348483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b78c97-6cd5-40dd-8c6a-e506e6602695 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:59.348757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b78c97-6cd5-40dd-8c6a-e506e6602695 None admin] PATH_INFO: `/v3/projects/ae270bf3b3a44357812b5125815645da` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:59.349971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b78c97-6cd5-40dd-8c6a-e506e6602695 None admin] RBAC: Authorizing `identity:get_project(project_id=ae270bf3b3a44357812b5125815645da)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:59.351405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b78c97-6cd5-40dd-8c6a-e506e6602695 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:59.353125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 577/1145] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:31:59 2022] GET /identity/v3/projects/ae270bf3b3a44357812b5125815645da => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:31:59.359973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a04ef-57ea-4122-8261-2a53cef2d1e9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:31:59.465735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a04ef-57ea-4122-8261-2a53cef2d1e9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:31:59.486530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:31:59.500989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2eea42-3762-4d43-9d82-c4792f153cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:31:59.501650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eea42-3762-4d43-9d82-c4792f153cab None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:31:59.501939 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eea42-3762-4d43-9d82-c4792f153cab None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:31:59.502213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eea42-3762-4d43-9d82-c4792f153cab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:31:59.503098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2eea42-3762-4d43-9d82-c4792f153cab None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=ae270bf3b3a44357812b5125815645da, user.id=806a72710a744fd1a7e9527fbe4f0d8b) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:31:59.503353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2eea42-3762-4d43-9d82-c4792f153cab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:31:59.504779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2eea42-3762-4d43-9d82-c4792f153cab None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:31:59.510284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 569/1146] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:31:59 2022] GET /identity/v3/role_assignments?user.id=806a72710a744fd1a7e9527fbe4f0d8b&scope.project.id=ae270bf3b3a44357812b5125815645da&role.id=81457691f3a2456fa33358cb6a922d05 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:01.502588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b1a90-9ff4-4dab-8a44-1c0b18bdc480 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:01.503170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b1a90-9ff4-4dab-8a44-1c0b18bdc480 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:01.503170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b1a90-9ff4-4dab-8a44-1c0b18bdc480 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:01.504507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 578/1147] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:01.509745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85eac80-2301-425e-b60f-8bd25777892e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:01.510177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85eac80-2301-425e-b60f-8bd25777892e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:01.510829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85eac80-2301-425e-b60f-8bd25777892e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:01.528155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d85eac80-2301-425e-b60f-8bd25777892e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:01.529379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d85eac80-2301-425e-b60f-8bd25777892e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:01.595856 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 570/1148] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:32:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:32:01.618791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b1a90-9ff4-4dab-8a44-1c0b18bdc480 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:01.628378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:01.720244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b1a90-9ff4-4dab-8a44-1c0b18bdc480 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:01.755564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96d73d-a2d5-4e8f-8ed7-b9c6ee85c6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:01.756278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96d73d-a2d5-4e8f-8ed7-b9c6ee85c6eb None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:01.756560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96d73d-a2d5-4e8f-8ed7-b9c6ee85c6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:01.756836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96d73d-a2d5-4e8f-8ed7-b9c6ee85c6eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:01.766967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96d73d-a2d5-4e8f-8ed7-b9c6ee85c6eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ce96d73d-a2d5-4e8f-8ed7-b9c6ee85c6eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:32:01.774579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:32:01.775725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:32:01.777110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 579/1149] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:32:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:32:01.781980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85eac80-2301-425e-b60f-8bd25777892e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:01.878855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85eac80-2301-425e-b60f-8bd25777892e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:01.892497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:01.912251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59793444-e058-4db1-a345-032d554bedab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:01.912954 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59793444-e058-4db1-a345-032d554bedab None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:01.913248 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59793444-e058-4db1-a345-032d554bedab None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:01.913525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59793444-e058-4db1-a345-032d554bedab None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:01.914272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59793444-e058-4db1-a345-032d554bedab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:32:01.914594 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59793444-e058-4db1-a345-032d554bedab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:01.916287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59793444-e058-4db1-a345-032d554bedab None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:01.925219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 571/1150] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:32:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:01.931087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96d73d-a2d5-4e8f-8ed7-b9c6ee85c6eb None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:01.997337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:02.032158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96d73d-a2d5-4e8f-8ed7-b9c6ee85c6eb None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:02.064478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464e3f3-2978-4bc5-b43d-a75053bd3795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:02.065428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e3f3-2978-4bc5-b43d-a75053bd3795 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:02.065707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e3f3-2978-4bc5-b43d-a75053bd3795 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:02.066932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e3f3-2978-4bc5-b43d-a75053bd3795 None admin] PATH_INFO: `/v3/users/806a72710a744fd1a7e9527fbe4f0d8b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:02.067159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464e3f3-2978-4bc5-b43d-a75053bd3795 None admin] RBAC: Authorizing `identity:get_user(user_id=806a72710a744fd1a7e9527fbe4f0d8b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:02.068580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464e3f3-2978-4bc5-b43d-a75053bd3795 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:02.070231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 580/1151] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:32:01 2022] GET /identity/v3/users/806a72710a744fd1a7e9527fbe4f0d8b => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:02.076640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59793444-e058-4db1-a345-032d554bedab None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:02.174762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59793444-e058-4db1-a345-032d554bedab None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:02.208288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85cf35-7712-40e3-9947-a739dc58e56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:02.209174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85cf35-7712-40e3-9947-a739dc58e56a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:02.209463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85cf35-7712-40e3-9947-a739dc58e56a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:02.209741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85cf35-7712-40e3-9947-a739dc58e56a None admin] PATH_INFO: `/v3/projects/ae270bf3b3a44357812b5125815645da` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:02.210925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85cf35-7712-40e3-9947-a739dc58e56a None admin] RBAC: Authorizing `identity:get_project(project_id=ae270bf3b3a44357812b5125815645da)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:02.212357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85cf35-7712-40e3-9947-a739dc58e56a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:02.214128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 572/1152] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:32:02 2022] GET /identity/v3/projects/ae270bf3b3a44357812b5125815645da => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:02.220412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464e3f3-2978-4bc5-b43d-a75053bd3795 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:02.319340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464e3f3-2978-4bc5-b43d-a75053bd3795 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:02.351656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9e7f9-bda2-4b65-9d9f-afec70fe65c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:02.352534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9e7f9-bda2-4b65-9d9f-afec70fe65c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:02.352811 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9e7f9-bda2-4b65-9d9f-afec70fe65c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:02.353116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9e7f9-bda2-4b65-9d9f-afec70fe65c7 None admin] PATH_INFO: `/v3/projects/ae270bf3b3a44357812b5125815645da/users/806a72710a744fd1a7e9527fbe4f0d8b/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:02.355582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9e7f9-bda2-4b65-9d9f-afec70fe65c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae270bf3b3a44357812b5125815645da, user_id=806a72710a744fd1a7e9527fbe4f0d8b, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:02.357046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9e7f9-bda2-4b65-9d9f-afec70fe65c7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:02.359300 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:02.470039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 581/1153] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:32:02 2022] PUT /identity/v3/projects/ae270bf3b3a44357812b5125815645da/users/806a72710a744fd1a7e9527fbe4f0d8b/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:32:04.451820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7d4b1-2f52-4a33-bab3-a5b11231435a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:04.452375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7d4b1-2f52-4a33-bab3-a5b11231435a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:04.452375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7d4b1-2f52-4a33-bab3-a5b11231435a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:04.453908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 573/1154] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:04.459229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5bc06-d74d-4411-ba43-a531ec125c1a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:04.459640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5bc06-d74d-4411-ba43-a531ec125c1a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:04.459784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5bc06-d74d-4411-ba43-a531ec125c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:04.476997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e6f5bc06-d74d-4411-ba43-a531ec125c1a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:04.478196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e6f5bc06-d74d-4411-ba43-a531ec125c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:04.554271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 582/1155] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:32:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:32:04.578047 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7d4b1-2f52-4a33-bab3-a5b11231435a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:04.627104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:04.683514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7d4b1-2f52-4a33-bab3-a5b11231435a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:04.717482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad55957-f995-4816-8243-8e7ae9fff458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:04.718205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad55957-f995-4816-8243-8e7ae9fff458 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:04.718491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad55957-f995-4816-8243-8e7ae9fff458 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:04.718768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad55957-f995-4816-8243-8e7ae9fff458 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:04.728817 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad55957-f995-4816-8243-8e7ae9fff458 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-7ad55957-f995-4816-8243-8e7ae9fff458 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:32:04.736953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:32:04.738167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:32:04.739207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 574/1156] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:32:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:32:04.744584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5bc06-d74d-4411-ba43-a531ec125c1a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:04.844538 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5bc06-d74d-4411-ba43-a531ec125c1a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:04.879104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:04.889737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065a233-82e2-4393-91ea-16cbdc269c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:04.890404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065a233-82e2-4393-91ea-16cbdc269c37 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:04.890690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065a233-82e2-4393-91ea-16cbdc269c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:04.890968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065a233-82e2-4393-91ea-16cbdc269c37 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:04.891827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065a233-82e2-4393-91ea-16cbdc269c37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:32:04.892138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065a233-82e2-4393-91ea-16cbdc269c37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:04.893564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065a233-82e2-4393-91ea-16cbdc269c37 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:04.902403 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 583/1157] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:32:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:04.908566 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad55957-f995-4816-8243-8e7ae9fff458 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:05.011798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad55957-f995-4816-8243-8e7ae9fff458 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:05.014592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:05.049051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c39e0-7443-42ee-87ed-ebad246fd168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:05.049920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c39e0-7443-42ee-87ed-ebad246fd168 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:05.050201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c39e0-7443-42ee-87ed-ebad246fd168 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:05.050477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c39e0-7443-42ee-87ed-ebad246fd168 None admin] PATH_INFO: `/v3/users/806a72710a744fd1a7e9527fbe4f0d8b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:05.051653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c39e0-7443-42ee-87ed-ebad246fd168 None admin] RBAC: Authorizing `identity:get_user(user_id=806a72710a744fd1a7e9527fbe4f0d8b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:05.053123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c39e0-7443-42ee-87ed-ebad246fd168 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:05.054759 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 575/1158] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:32:04 2022] GET /identity/v3/users/806a72710a744fd1a7e9527fbe4f0d8b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:05.060582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065a233-82e2-4393-91ea-16cbdc269c37 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:05.162830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065a233-82e2-4393-91ea-16cbdc269c37 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:05.200809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c30a1-87f7-4f70-8ef0-fb80e38cf440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:05.201846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c30a1-87f7-4f70-8ef0-fb80e38cf440 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:05.202268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c30a1-87f7-4f70-8ef0-fb80e38cf440 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:05.202670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c30a1-87f7-4f70-8ef0-fb80e38cf440 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:05.264454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:05.277582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c30a1-87f7-4f70-8ef0-fb80e38cf440 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:05.279038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c30a1-87f7-4f70-8ef0-fb80e38cf440 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:05.374204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 584/1159] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:32:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:32:05.380120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c39e0-7443-42ee-87ed-ebad246fd168 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:05.480239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c39e0-7443-42ee-87ed-ebad246fd168 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:05.514095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cff65-3b39-44aa-be6a-bc60009afa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:05.514994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cff65-3b39-44aa-be6a-bc60009afa10 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:05.515225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cff65-3b39-44aa-be6a-bc60009afa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:05.515498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cff65-3b39-44aa-be6a-bc60009afa10 None admin] PATH_INFO: `/v3/projects/ae270bf3b3a44357812b5125815645da` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:05.516778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cff65-3b39-44aa-be6a-bc60009afa10 None admin] RBAC: Authorizing `identity:get_project(project_id=ae270bf3b3a44357812b5125815645da)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:05.518251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cff65-3b39-44aa-be6a-bc60009afa10 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:05.519953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 576/1160] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:32:05 2022] GET /identity/v3/projects/ae270bf3b3a44357812b5125815645da => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:05.527047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c30a1-87f7-4f70-8ef0-fb80e38cf440 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:05.611905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:05.628113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c30a1-87f7-4f70-8ef0-fb80e38cf440 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:05.662134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63657b9-5741-4ab9-9faa-0c400f5badd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:05.662769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63657b9-5741-4ab9-9faa-0c400f5badd8 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:05.663093 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63657b9-5741-4ab9-9faa-0c400f5badd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:05.663319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63657b9-5741-4ab9-9faa-0c400f5badd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:05.664164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63657b9-5741-4ab9-9faa-0c400f5badd8 None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=ae270bf3b3a44357812b5125815645da, user.id=806a72710a744fd1a7e9527fbe4f0d8b) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:32:05.664463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63657b9-5741-4ab9-9faa-0c400f5badd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:05.665913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63657b9-5741-4ab9-9faa-0c400f5badd8 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:05.671932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 585/1161] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:32:05 2022] GET /identity/v3/role_assignments?user.id=806a72710a744fd1a7e9527fbe4f0d8b&scope.project.id=ae270bf3b3a44357812b5125815645da&role.id=81457691f3a2456fa33358cb6a922d05 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:07.710970 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702ec45-dd5f-4314-bcd3-5900a5c8d2df None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:07.711551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702ec45-dd5f-4314-bcd3-5900a5c8d2df None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:07.711551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702ec45-dd5f-4314-bcd3-5900a5c8d2df None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:07.712937 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 577/1162] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:07.718514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59cc3-4b22-4c06-a856-efb4c66ea9e4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:07.718970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59cc3-4b22-4c06-a856-efb4c66ea9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:07.719072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59cc3-4b22-4c06-a856-efb4c66ea9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:07.736459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bbd59cc3-4b22-4c06-a856-efb4c66ea9e4 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:07.737606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bbd59cc3-4b22-4c06-a856-efb4c66ea9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:07.804562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 586/1163] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:32:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:32:07.827482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702ec45-dd5f-4314-bcd3-5900a5c8d2df None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:07.872445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:07.933055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702ec45-dd5f-4314-bcd3-5900a5c8d2df None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:07.969659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2773b2-5bad-4bd5-a304-87f3985ef44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:07.970222 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2773b2-5bad-4bd5-a304-87f3985ef44c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:07.970515 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2773b2-5bad-4bd5-a304-87f3985ef44c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:07.970793 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2773b2-5bad-4bd5-a304-87f3985ef44c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:08.040673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2773b2-5bad-4bd5-a304-87f3985ef44c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:08.042742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2773b2-5bad-4bd5-a304-87f3985ef44c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:08.083471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:08.143593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 578/1164] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:32:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:32:08.149566 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd59cc3-4b22-4c06-a856-efb4c66ea9e4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:08.236037 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:08.256754 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd59cc3-4b22-4c06-a856-efb4c66ea9e4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:08.291496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a658f2-87b9-45f0-88cc-5af518bc5c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:08.292326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a658f2-87b9-45f0-88cc-5af518bc5c82 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:08.292648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a658f2-87b9-45f0-88cc-5af518bc5c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:08.292951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a658f2-87b9-45f0-88cc-5af518bc5c82 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:08.299665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a658f2-87b9-45f0-88cc-5af518bc5c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:08.301127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a658f2-87b9-45f0-88cc-5af518bc5c82 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-16a658f2-87b9-45f0-88cc-5af518bc5c82 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:32:08.316540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:32:08.317807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:32:08.318866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 587/1165] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Jul 18 16:32:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:32:08.324512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2773b2-5bad-4bd5-a304-87f3985ef44c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:08.423953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2773b2-5bad-4bd5-a304-87f3985ef44c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:08.458366 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733b6db-72cb-4955-a833-43396c142320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:08.459025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733b6db-72cb-4955-a833-43396c142320 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:08.459308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733b6db-72cb-4955-a833-43396c142320 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:08.459582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733b6db-72cb-4955-a833-43396c142320 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:08.460303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733b6db-72cb-4955-a833-43396c142320 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:32:08.460607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733b6db-72cb-4955-a833-43396c142320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:08.462004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733b6db-72cb-4955-a833-43396c142320 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:08.469676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 579/1166] 10.222.0.88 () {56 vars in 1115 bytes} [Mon Jul 18 16:32:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:10.449156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a20d5d-a6ba-4ee7-9989-ea55be715b24 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:10.449722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a20d5d-a6ba-4ee7-9989-ea55be715b24 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:10.449722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a20d5d-a6ba-4ee7-9989-ea55be715b24 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:10.451079 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 588/1167] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:10.456900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc70d6-749f-4546-8713-91317cb15a59 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:10.457321 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc70d6-749f-4546-8713-91317cb15a59 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:10.457466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc70d6-749f-4546-8713-91317cb15a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:10.463428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:10.473965 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-cdbc70d6-749f-4546-8713-91317cb15a59 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:10.475156 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cdbc70d6-749f-4546-8713-91317cb15a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:10.542401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 580/1168] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:32:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:32:10.565404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a20d5d-a6ba-4ee7-9989-ea55be715b24 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:10.667019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a20d5d-a6ba-4ee7-9989-ea55be715b24 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:10.700480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cc6ac-5454-4758-b828-ef9beafa40a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:10.701086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cc6ac-5454-4758-b828-ef9beafa40a4 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:10.701374 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cc6ac-5454-4758-b828-ef9beafa40a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:10.701658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cc6ac-5454-4758-b828-ef9beafa40a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:10.702447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2cc6ac-5454-4758-b828-ef9beafa40a4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:10.703832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2cc6ac-5454-4758-b828-ef9beafa40a4 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:10.894717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:10.901804 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 589/1169] 10.222.0.88 () {60 vars in 1138 bytes} [Mon Jul 18 16:32:10 2022] POST /identity/v3/domains => generated 267 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:32:12.933078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b115e5-d247-4cad-8c43-514a7063ea03 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:12.933505 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b115e5-d247-4cad-8c43-514a7063ea03 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:12.933648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b115e5-d247-4cad-8c43-514a7063ea03 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:12.935060 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 581/1170] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:12.940566 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a715238-9368-49a8-b756-bc194adf2826 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:12.940997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a715238-9368-49a8-b756-bc194adf2826 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:12.941266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a715238-9368-49a8-b756-bc194adf2826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:12.958548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1a715238-9368-49a8-b756-bc194adf2826 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:12.959737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1a715238-9368-49a8-b756-bc194adf2826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:13.027421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 590/1171] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:32:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:32:13.051032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b115e5-d247-4cad-8c43-514a7063ea03 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:13.153022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b115e5-d247-4cad-8c43-514a7063ea03 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:13.188322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1f859-7e05-4304-a0e8-96ebb4ec71ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:13.189131 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1f859-7e05-4304-a0e8-96ebb4ec71ff None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:13.189414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1f859-7e05-4304-a0e8-96ebb4ec71ff None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:13.189685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1f859-7e05-4304-a0e8-96ebb4ec71ff None admin] PATH_INFO: `/v3/domains/e242a5feebba42fcaf6de36949bc7ae6` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:13.191635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:13.201388 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1f859-7e05-4304-a0e8-96ebb4ec71ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=e242a5feebba42fcaf6de36949bc7ae6)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:13.202838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1f859-7e05-4304-a0e8-96ebb4ec71ff None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:13.211122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 582/1172] 10.222.0.88 () {56 vars in 1183 bytes} [Mon Jul 18 16:32:13 2022] GET /identity/v3/domains/e242a5feebba42fcaf6de36949bc7ae6 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:13.211253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a715238-9368-49a8-b756-bc194adf2826 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:13.309987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a715238-9368-49a8-b756-bc194adf2826 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:13.337997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:13.344157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76368366-c2b0-403b-96d2-f2357c3c0e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:13.344809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76368366-c2b0-403b-96d2-f2357c3c0e5b None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:13.348653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76368366-c2b0-403b-96d2-f2357c3c0e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:13.348653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76368366-c2b0-403b-96d2-f2357c3c0e5b None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:13.348653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76368366-c2b0-403b-96d2-f2357c3c0e5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:13.348653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76368366-c2b0-403b-96d2-f2357c3c0e5b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:13.442124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 591/1173] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:32:13 2022] POST /identity/v3/projects => generated 361 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:32:15.476861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62008-05f2-4fa8-8150-da1c95fd1ba9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:15.477410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62008-05f2-4fa8-8150-da1c95fd1ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:15.477514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62008-05f2-4fa8-8150-da1c95fd1ba9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:15.478845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 583/1174] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:15.484075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d93ce3-348f-44e1-9b2c-058db0cf604b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:15.484497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d93ce3-348f-44e1-9b2c-058db0cf604b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:15.484638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d93ce3-348f-44e1-9b2c-058db0cf604b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:15.501986 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e4d93ce3-348f-44e1-9b2c-058db0cf604b None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:15.503179 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e4d93ce3-348f-44e1-9b2c-058db0cf604b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:15.568454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 592/1175] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:32:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:32:15.592824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b62008-05f2-4fa8-8150-da1c95fd1ba9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:15.694213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b62008-05f2-4fa8-8150-da1c95fd1ba9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:15.732910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9f122-767b-4683-920c-98b3b32761c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:15.734121 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9f122-767b-4683-920c-98b3b32761c0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:15.734536 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9f122-767b-4683-920c-98b3b32761c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:15.734940 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9f122-767b-4683-920c-98b3b32761c0 None admin] PATH_INFO: `/v3/domains/e242a5feebba42fcaf6de36949bc7ae6` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:15.736604 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9f122-767b-4683-920c-98b3b32761c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e242a5feebba42fcaf6de36949bc7ae6)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:15.738830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9f122-767b-4683-920c-98b3b32761c0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:15.741595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 584/1176] 10.222.0.88 () {56 vars in 1183 bytes} [Mon Jul 18 16:32:15 2022] GET /identity/v3/domains/e242a5feebba42fcaf6de36949bc7ae6 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:15.748109 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d93ce3-348f-44e1-9b2c-058db0cf604b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:15.849045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d93ce3-348f-44e1-9b2c-058db0cf604b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:15.862871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:15.884548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565dc5c-a0c2-40d8-b4e4-e8f78dc48910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:15.885289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565dc5c-a0c2-40d8-b4e4-e8f78dc48910 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:15.885571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565dc5c-a0c2-40d8-b4e4-e8f78dc48910 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:15.885846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565dc5c-a0c2-40d8-b4e4-e8f78dc48910 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:15.886551 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565dc5c-a0c2-40d8-b4e4-e8f78dc48910 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:15.887985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565dc5c-a0c2-40d8-b4e4-e8f78dc48910 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:16.025385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 593/1177] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:32:15 2022] POST /identity/v3/users => generated 311 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:32:18.102533 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ec1c6-ac78-4cde-a378-c411ce138892 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:18.103051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ec1c6-ac78-4cde-a378-c411ce138892 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:18.103139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ec1c6-ac78-4cde-a378-c411ce138892 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:18.104495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 585/1178] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:18.109894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4341b0-8345-49de-b085-ae00bfe3479a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:18.110289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4341b0-8345-49de-b085-ae00bfe3479a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:18.110458 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4341b0-8345-49de-b085-ae00bfe3479a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:18.127396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3b4341b0-8345-49de-b085-ae00bfe3479a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:18.128596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3b4341b0-8345-49de-b085-ae00bfe3479a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:18.198850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 594/1179] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:32:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:32:18.222052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ec1c6-ac78-4cde-a378-c411ce138892 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:18.284257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:18.323148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ec1c6-ac78-4cde-a378-c411ce138892 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:18.357327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00d178-bc4d-4734-92f1-bcf122bb7e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:18.358120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00d178-bc4d-4734-92f1-bcf122bb7e08 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:18.358408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00d178-bc4d-4734-92f1-bcf122bb7e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:18.358686 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00d178-bc4d-4734-92f1-bcf122bb7e08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:18.369130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae00d178-bc4d-4734-92f1-bcf122bb7e08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-ae00d178-bc4d-4734-92f1-bcf122bb7e08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:32:18.377999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:32:18.379148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:32:18.380200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 586/1180] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:32:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:32:18.386329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4341b0-8345-49de-b085-ae00bfe3479a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:18.482364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:18.486029 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4341b0-8345-49de-b085-ae00bfe3479a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:18.519954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5c98d-78ed-47d4-987a-b82652612f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:18.520630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c98d-78ed-47d4-987a-b82652612f1a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:18.521148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c98d-78ed-47d4-987a-b82652612f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:18.521640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5c98d-78ed-47d4-987a-b82652612f1a None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:18.522777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5c98d-78ed-47d4-987a-b82652612f1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:32:18.523143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5c98d-78ed-47d4-987a-b82652612f1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:18.524517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5c98d-78ed-47d4-987a-b82652612f1a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:18.532552 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 595/1181] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:32:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:18.538515 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00d178-bc4d-4734-92f1-bcf122bb7e08 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:18.638184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00d178-bc4d-4734-92f1-bcf122bb7e08 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:18.660334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:18.671668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4d023-4197-4d02-84a3-00035a544bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:18.678835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4d023-4197-4d02-84a3-00035a544bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:18.678835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4d023-4197-4d02-84a3-00035a544bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:18.678835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4d023-4197-4d02-84a3-00035a544bd6 None admin] PATH_INFO: `/v3/users/fc0bf90529e84c559b562ce2c078ce5e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:18.686583 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4d023-4197-4d02-84a3-00035a544bd6 None admin] RBAC: Authorizing `identity:get_user(user_id=fc0bf90529e84c559b562ce2c078ce5e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:18.688030 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4d023-4197-4d02-84a3-00035a544bd6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:18.689747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 587/1182] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:32:18 2022] GET /identity/v3/users/fc0bf90529e84c559b562ce2c078ce5e => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:18.695715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5c98d-78ed-47d4-987a-b82652612f1a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:18.795329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5c98d-78ed-47d4-987a-b82652612f1a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:18.828708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc39c1f-1d79-4615-b37f-4984a2dce1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:18.829384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc39c1f-1d79-4615-b37f-4984a2dce1bd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:18.829666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc39c1f-1d79-4615-b37f-4984a2dce1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:18.829945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc39c1f-1d79-4615-b37f-4984a2dce1bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:18.858055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:18.897794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc39c1f-1d79-4615-b37f-4984a2dce1bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:18.899260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc39c1f-1d79-4615-b37f-4984a2dce1bd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:19.001287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 596/1183] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:32:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:32:19.007252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4d023-4197-4d02-84a3-00035a544bd6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:19.107696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4d023-4197-4d02-84a3-00035a544bd6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:19.144923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5342b0-5268-4b6c-8f0a-5b8cdd2c33d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:19.145794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5342b0-5268-4b6c-8f0a-5b8cdd2c33d2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:19.146078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5342b0-5268-4b6c-8f0a-5b8cdd2c33d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:19.146357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5342b0-5268-4b6c-8f0a-5b8cdd2c33d2 None admin] PATH_INFO: `/v3/projects/1a7759789846490b87fa391818b914bf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:19.147557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5342b0-5268-4b6c-8f0a-5b8cdd2c33d2 None admin] RBAC: Authorizing `identity:get_project(project_id=1a7759789846490b87fa391818b914bf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:19.149024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5342b0-5268-4b6c-8f0a-5b8cdd2c33d2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:19.151293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 588/1184] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:32:19 2022] GET /identity/v3/projects/1a7759789846490b87fa391818b914bf => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:19.157727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc39c1f-1d79-4615-b37f-4984a2dce1bd None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:19.216352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:19.257709 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc39c1f-1d79-4615-b37f-4984a2dce1bd None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:19.293023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae203ce3-ee84-4543-b9fc-e0811df18a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:19.293680 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae203ce3-ee84-4543-b9fc-e0811df18a0e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:19.293961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae203ce3-ee84-4543-b9fc-e0811df18a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:19.294237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae203ce3-ee84-4543-b9fc-e0811df18a0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:19.295074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae203ce3-ee84-4543-b9fc-e0811df18a0e None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=1a7759789846490b87fa391818b914bf, user.id=fc0bf90529e84c559b562ce2c078ce5e) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:32:19.295385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae203ce3-ee84-4543-b9fc-e0811df18a0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:19.296834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae203ce3-ee84-4543-b9fc-e0811df18a0e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:19.302345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 597/1185] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:32:19 2022] GET /identity/v3/role_assignments?user.id=fc0bf90529e84c559b562ce2c078ce5e&scope.project.id=1a7759789846490b87fa391818b914bf&role.id=81457691f3a2456fa33358cb6a922d05 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:21.307964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bce848-6a80-40b0-88ef-e1b8368cc786 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:21.308553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bce848-6a80-40b0-88ef-e1b8368cc786 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:21.308553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bce848-6a80-40b0-88ef-e1b8368cc786 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:21.309961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 589/1186] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:21.315008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea8d17-edf1-4e5d-855d-6312cd89a2b8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:21.315518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea8d17-edf1-4e5d-855d-6312cd89a2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:21.315622 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea8d17-edf1-4e5d-855d-6312cd89a2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:21.332469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-baea8d17-edf1-4e5d-855d-6312cd89a2b8 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:21.333685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-baea8d17-edf1-4e5d-855d-6312cd89a2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:21.400016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 598/1187] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:32:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:32:21.422497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bce848-6a80-40b0-88ef-e1b8368cc786 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:21.472948 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:21.524445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bce848-6a80-40b0-88ef-e1b8368cc786 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:21.557569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569c2e9-6906-4249-af2d-e137a400404b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:21.558392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569c2e9-6906-4249-af2d-e137a400404b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:21.558624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569c2e9-6906-4249-af2d-e137a400404b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:21.558892 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569c2e9-6906-4249-af2d-e137a400404b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:21.568910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569c2e9-6906-4249-af2d-e137a400404b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a569c2e9-6906-4249-af2d-e137a400404b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:32:21.576985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:32:21.578304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:32:21.579335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 590/1188] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:32:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:32:21.584301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea8d17-edf1-4e5d-855d-6312cd89a2b8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:21.639929 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:21.684155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea8d17-edf1-4e5d-855d-6312cd89a2b8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:21.719989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6b241-11b9-4ef0-b559-f45437e3d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:21.720658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6b241-11b9-4ef0-b559-f45437e3d725 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:21.720973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6b241-11b9-4ef0-b559-f45437e3d725 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:21.721253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6b241-11b9-4ef0-b559-f45437e3d725 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:21.723955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6b241-11b9-4ef0-b559-f45437e3d725 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:32:21.724286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6b241-11b9-4ef0-b559-f45437e3d725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:21.725785 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6b241-11b9-4ef0-b559-f45437e3d725 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:21.735306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 599/1189] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:32:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:21.741461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569c2e9-6906-4249-af2d-e137a400404b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:21.843632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569c2e9-6906-4249-af2d-e137a400404b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:21.852936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:21.880174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366bf26-917b-419d-8118-cedd6a358259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:21.881095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366bf26-917b-419d-8118-cedd6a358259 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:21.881378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366bf26-917b-419d-8118-cedd6a358259 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:21.881650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366bf26-917b-419d-8118-cedd6a358259 None admin] PATH_INFO: `/v3/users/fc0bf90529e84c559b562ce2c078ce5e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:21.882911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366bf26-917b-419d-8118-cedd6a358259 None admin] RBAC: Authorizing `identity:get_user(user_id=fc0bf90529e84c559b562ce2c078ce5e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:21.884343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366bf26-917b-419d-8118-cedd6a358259 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:21.886462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 591/1190] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:32:21 2022] GET /identity/v3/users/fc0bf90529e84c559b562ce2c078ce5e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:21.892433 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6b241-11b9-4ef0-b559-f45437e3d725 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:22.005363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6b241-11b9-4ef0-b559-f45437e3d725 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:22.021539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:22.044085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4c69f-6d71-49c9-9140-2e2276d7423c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:22.044993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c69f-6d71-49c9-9140-2e2276d7423c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:22.045278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c69f-6d71-49c9-9140-2e2276d7423c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:22.045557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4c69f-6d71-49c9-9140-2e2276d7423c None admin] PATH_INFO: `/v3/projects/1a7759789846490b87fa391818b914bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:22.046778 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4c69f-6d71-49c9-9140-2e2276d7423c None admin] RBAC: Authorizing `identity:get_project(project_id=1a7759789846490b87fa391818b914bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:22.048407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4c69f-6d71-49c9-9140-2e2276d7423c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:22.050152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 600/1191] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:32:21 2022] GET /identity/v3/projects/1a7759789846490b87fa391818b914bf => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:22.056605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366bf26-917b-419d-8118-cedd6a358259 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:22.160951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366bf26-917b-419d-8118-cedd6a358259 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:22.196575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c3472-db49-405e-bbb8-1b604c565927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:22.197550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c3472-db49-405e-bbb8-1b604c565927 None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:22.197838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c3472-db49-405e-bbb8-1b604c565927 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:22.198293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c3472-db49-405e-bbb8-1b604c565927 None admin] PATH_INFO: `/v3/projects/1a7759789846490b87fa391818b914bf/users/fc0bf90529e84c559b562ce2c078ce5e/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:22.200519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c3472-db49-405e-bbb8-1b604c565927 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a7759789846490b87fa391818b914bf, user_id=fc0bf90529e84c559b562ce2c078ce5e, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:22.202000 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c3472-db49-405e-bbb8-1b604c565927 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:22.276820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 592/1192] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:32:22 2022] PUT /identity/v3/projects/1a7759789846490b87fa391818b914bf/users/fc0bf90529e84c559b562ce2c078ce5e/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:32:24.246904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b67e9-eed2-4c7e-b556-667cf5fc89ff None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:24.247439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b67e9-eed2-4c7e-b556-667cf5fc89ff None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:24.247439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b67e9-eed2-4c7e-b556-667cf5fc89ff None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:24.248741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 601/1193] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:32:24.254090 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d207bd-1ccb-4a5d-b4a8-e26d50613c54 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:24.254524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d207bd-1ccb-4a5d-b4a8-e26d50613c54 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:24.254644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d207bd-1ccb-4a5d-b4a8-e26d50613c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:24.271518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-31d207bd-1ccb-4a5d-b4a8-e26d50613c54 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:32:24.272687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-31d207bd-1ccb-4a5d-b4a8-e26d50613c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:32:24.301448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:24.340830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 593/1194] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:32:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:32:24.363500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b67e9-eed2-4c7e-b556-667cf5fc89ff None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:24.469964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b67e9-eed2-4c7e-b556-667cf5fc89ff None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:24.502951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68ee79-db75-423e-a4ef-f9eaa273674b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:24.503731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68ee79-db75-423e-a4ef-f9eaa273674b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:24.504010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68ee79-db75-423e-a4ef-f9eaa273674b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:24.504281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68ee79-db75-423e-a4ef-f9eaa273674b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:24.515887 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68ee79-db75-423e-a4ef-f9eaa273674b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-8f68ee79-db75-423e-a4ef-f9eaa273674b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:32:24.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:32:24.524456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:32:24.525579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 602/1195] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:32:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:32:24.530950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d207bd-1ccb-4a5d-b4a8-e26d50613c54 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:24.627707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d207bd-1ccb-4a5d-b4a8-e26d50613c54 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:24.661352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba59bc-1ce4-40d4-8f5f-1f902afffe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:24.662022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba59bc-1ce4-40d4-8f5f-1f902afffe24 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:24.662310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba59bc-1ce4-40d4-8f5f-1f902afffe24 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:24.662590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba59bc-1ce4-40d4-8f5f-1f902afffe24 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:24.663339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba59bc-1ce4-40d4-8f5f-1f902afffe24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:32:24.663645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba59bc-1ce4-40d4-8f5f-1f902afffe24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:24.665041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ba59bc-1ce4-40d4-8f5f-1f902afffe24 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:24.673054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 594/1196] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:32:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:24.678924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68ee79-db75-423e-a4ef-f9eaa273674b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:24.725181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:24.779232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68ee79-db75-423e-a4ef-f9eaa273674b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:24.813166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc716dc-a06c-49a0-9580-2cc0dd72952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:24.814023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc716dc-a06c-49a0-9580-2cc0dd72952d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:24.814325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc716dc-a06c-49a0-9580-2cc0dd72952d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:24.814587 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc716dc-a06c-49a0-9580-2cc0dd72952d None admin] PATH_INFO: `/v3/users/fc0bf90529e84c559b562ce2c078ce5e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:24.815780 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc716dc-a06c-49a0-9580-2cc0dd72952d None admin] RBAC: Authorizing `identity:get_user(user_id=fc0bf90529e84c559b562ce2c078ce5e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:24.817248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc716dc-a06c-49a0-9580-2cc0dd72952d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:24.818880 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 603/1197] 10.222.0.88 () {56 vars in 1177 bytes} [Mon Jul 18 16:32:24 2022] GET /identity/v3/users/fc0bf90529e84c559b562ce2c078ce5e => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:24.824984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba59bc-1ce4-40d4-8f5f-1f902afffe24 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:24.853073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:24.923669 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba59bc-1ce4-40d4-8f5f-1f902afffe24 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:24.958193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe4770-05f5-4e65-9ee8-666d44b8dc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:24.958841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe4770-05f5-4e65-9ee8-666d44b8dc7d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:24.959125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe4770-05f5-4e65-9ee8-666d44b8dc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:24.959401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe4770-05f5-4e65-9ee8-666d44b8dc7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:25.025442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe4770-05f5-4e65-9ee8-666d44b8dc7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:25.026921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe4770-05f5-4e65-9ee8-666d44b8dc7d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:25.053355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:25.124621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 595/1198] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:32:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:32:25.130645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc716dc-a06c-49a0-9580-2cc0dd72952d None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:25.232352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc716dc-a06c-49a0-9580-2cc0dd72952d None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:25.245980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:25.266325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1eb1e1-ec6b-4255-9d8e-7296be1b92e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:25.267178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1eb1e1-ec6b-4255-9d8e-7296be1b92e5 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:25.267458 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1eb1e1-ec6b-4255-9d8e-7296be1b92e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:25.267732 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1eb1e1-ec6b-4255-9d8e-7296be1b92e5 None admin] PATH_INFO: `/v3/projects/1a7759789846490b87fa391818b914bf` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:25.268923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1eb1e1-ec6b-4255-9d8e-7296be1b92e5 None admin] RBAC: Authorizing `identity:get_project(project_id=1a7759789846490b87fa391818b914bf)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:25.270380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1eb1e1-ec6b-4255-9d8e-7296be1b92e5 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:25.272065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 604/1199] 10.222.0.88 () {56 vars in 1186 bytes} [Mon Jul 18 16:32:25 2022] GET /identity/v3/projects/1a7759789846490b87fa391818b914bf => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:25.278975 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe4770-05f5-4e65-9ee8-666d44b8dc7d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:32:25.379876 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe4770-05f5-4e65-9ee8-666d44b8dc7d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:32:25.405690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:25.413566 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d28ef-d521-41d7-97e4-db07bbac274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:32:25.414226 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d28ef-d521-41d7-97e4-db07bbac274a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:32:25.414510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d28ef-d521-41d7-97e4-db07bbac274a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:32:25.414791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d28ef-d521-41d7-97e4-db07bbac274a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:32:25.415621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d28ef-d521-41d7-97e4-db07bbac274a None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=1a7759789846490b87fa391818b914bf, user.id=fc0bf90529e84c559b562ce2c078ce5e) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:32:25.416618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d28ef-d521-41d7-97e4-db07bbac274a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:32:25.418108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d28ef-d521-41d7-97e4-db07bbac274a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:32:25.424399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 596/1200] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:32:25 2022] GET /identity/v3/role_assignments?user.id=fc0bf90529e84c559b562ce2c078ce5e&scope.project.id=1a7759789846490b87fa391818b914bf&role.id=81457691f3a2456fa33358cb6a922d05 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:32:35.273558 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=10 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:32:35.447837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:40.667794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014f9c-f79c-41aa-b948-c701ea30ad50 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:40.668440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014f9c-f79c-41aa-b948-c701ea30ad50 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:40.668440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72014f9c-f79c-41aa-b948-c701ea30ad50 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:40.669764 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 605/1201] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:33:40.674984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e9573-eadb-4800-aa99-f7cb114263c7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:40.675445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e9573-eadb-4800-aa99-f7cb114263c7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:40.675550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e9573-eadb-4800-aa99-f7cb114263c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:40.692645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6a5e9573-eadb-4800-aa99-f7cb114263c7 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:40.694461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6a5e9573-eadb-4800-aa99-f7cb114263c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:40.762755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 597/1202] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:33:40.787239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72014f9c-f79c-41aa-b948-c701ea30ad50 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:40.891984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72014f9c-f79c-41aa-b948-c701ea30ad50 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:40.925159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb5893-75bf-47ec-8c17-e47a879a445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:40.925894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb5893-75bf-47ec-8c17-e47a879a445b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:40.926177 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb5893-75bf-47ec-8c17-e47a879a445b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:40.926453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb5893-75bf-47ec-8c17-e47a879a445b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:40.939431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb5893-75bf-47ec-8c17-e47a879a445b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:40.939431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb5893-75bf-47ec-8c17-e47a879a445b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-21cb5893-75bf-47ec-8c17-e47a879a445b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:33:40.957234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:40.958397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:40.959645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:33:40.959645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:33:40.959645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:40.959645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:33:40.959645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 606/1203] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:40.964537 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e9573-eadb-4800-aa99-f7cb114263c7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:41.063477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e9573-eadb-4800-aa99-f7cb114263c7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:41.092035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:41.097560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf92c9-7855-4196-8921-1c8aa2b4a099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:41.098222 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf92c9-7855-4196-8921-1c8aa2b4a099 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:41.098507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf92c9-7855-4196-8921-1c8aa2b4a099 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:41.098782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf92c9-7855-4196-8921-1c8aa2b4a099 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:41.099502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf92c9-7855-4196-8921-1c8aa2b4a099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:41.099807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf92c9-7855-4196-8921-1c8aa2b4a099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:41.101214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf92c9-7855-4196-8921-1c8aa2b4a099 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:41.112256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 598/1204] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:41.119039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb5893-75bf-47ec-8c17-e47a879a445b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:41.171557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:41.220227 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb5893-75bf-47ec-8c17-e47a879a445b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:41.254206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06558e-46c8-499d-b287-71ce86d7ee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:41.254866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06558e-46c8-499d-b287-71ce86d7ee08 None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:41.255151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06558e-46c8-499d-b287-71ce86d7ee08 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:41.255429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06558e-46c8-499d-b287-71ce86d7ee08 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:41.256129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06558e-46c8-499d-b287-71ce86d7ee08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:41.257600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06558e-46c8-499d-b287-71ce86d7ee08 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:41.395651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 607/1205] 10.222.0.88 () {60 vars in 1132 bytes} [Mon Jul 18 16:33:41 2022] POST /identity/v3/users => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:33:43.457261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30566d2f-fd7b-4e5f-bffa-5fe3029bd658 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:43.457789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30566d2f-fd7b-4e5f-bffa-5fe3029bd658 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:43.457877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30566d2f-fd7b-4e5f-bffa-5fe3029bd658 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:43.459209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 599/1206] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:33:43.464463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14240553-a691-4dbf-ad86-fc5bd5537895 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:43.464939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14240553-a691-4dbf-ad86-fc5bd5537895 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:43.465057 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14240553-a691-4dbf-ad86-fc5bd5537895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:43.482262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-14240553-a691-4dbf-ad86-fc5bd5537895 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:43.483439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-14240553-a691-4dbf-ad86-fc5bd5537895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:43.551818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 608/1207] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:33:43.575748 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30566d2f-fd7b-4e5f-bffa-5fe3029bd658 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:43.678105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30566d2f-fd7b-4e5f-bffa-5fe3029bd658 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:43.713619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9008d9c-5467-429c-9d73-a26345cc18dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:43.714335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008d9c-5467-429c-9d73-a26345cc18dd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:43.714618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008d9c-5467-429c-9d73-a26345cc18dd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:43.714891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9008d9c-5467-429c-9d73-a26345cc18dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:43.730375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9008d9c-5467-429c-9d73-a26345cc18dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-f9008d9c-5467-429c-9d73-a26345cc18dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:33:43.738861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:43.740298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:43.741372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 600/1208] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:33:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:43.746660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14240553-a691-4dbf-ad86-fc5bd5537895 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:43.787318 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:43.847755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14240553-a691-4dbf-ad86-fc5bd5537895 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:43.881519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf5b45-c0e8-4da3-89ab-48637cc47b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:43.882185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf5b45-c0e8-4da3-89ab-48637cc47b76 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:43.882469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf5b45-c0e8-4da3-89ab-48637cc47b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:43.882747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf5b45-c0e8-4da3-89ab-48637cc47b76 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:43.883498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf5b45-c0e8-4da3-89ab-48637cc47b76 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:43.883803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf5b45-c0e8-4da3-89ab-48637cc47b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:43.885236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf5b45-c0e8-4da3-89ab-48637cc47b76 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:43.893390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 609/1209] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:33:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:43.899087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9008d9c-5467-429c-9d73-a26345cc18dd None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:43.952828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:44.003439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9008d9c-5467-429c-9d73-a26345cc18dd None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:44.036387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826bd08-0aca-48b1-a8b8-6e0e777c222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:44.037213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826bd08-0aca-48b1-a8b8-6e0e777c222b None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:44.037496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826bd08-0aca-48b1-a8b8-6e0e777c222b None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:44.037769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826bd08-0aca-48b1-a8b8-6e0e777c222b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:44.049219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826bd08-0aca-48b1-a8b8-6e0e777c222b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:44.055531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826bd08-0aca-48b1-a8b8-6e0e777c222b None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-2826bd08-0aca-48b1-a8b8-6e0e777c222b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:33:44.067486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:44.069021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:44.069960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:33:44.069960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:33:44.069960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:44.069960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:44.069960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 601/1210] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:44.074569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf5b45-c0e8-4da3-89ab-48637cc47b76 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:44.176746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:44.180264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf5b45-c0e8-4da3-89ab-48637cc47b76 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:44.213808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2a5e7-77ee-4dfa-8f3c-44c0e9386611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:44.214550 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2a5e7-77ee-4dfa-8f3c-44c0e9386611 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:44.214743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2a5e7-77ee-4dfa-8f3c-44c0e9386611 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:44.215013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2a5e7-77ee-4dfa-8f3c-44c0e9386611 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:44.215728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2a5e7-77ee-4dfa-8f3c-44c0e9386611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:44.216064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2a5e7-77ee-4dfa-8f3c-44c0e9386611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:44.217478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2a5e7-77ee-4dfa-8f3c-44c0e9386611 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:44.229655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 610/1211] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:44.235473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826bd08-0aca-48b1-a8b8-6e0e777c222b None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:44.339504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826bd08-0aca-48b1-a8b8-6e0e777c222b None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:44.340996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:44.374009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13066e05-3622-4730-9f63-c6eca602e244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:44.374830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13066e05-3622-4730-9f63-c6eca602e244 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:44.375157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13066e05-3622-4730-9f63-c6eca602e244 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:44.375440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13066e05-3622-4730-9f63-c6eca602e244 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:44.382671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13066e05-3622-4730-9f63-c6eca602e244 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:44.387368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13066e05-3622-4730-9f63-c6eca602e244 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-13066e05-3622-4730-9f63-c6eca602e244 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:33:44.399739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:44.401086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:44.402251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:44.402251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:44.402251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:33:44.402251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:33:44.402251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:33:44.402251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:33:44.402251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:33:44.402251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:33:44.402251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:33:44.402251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:44.402251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 602/1212] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:44.407641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2a5e7-77ee-4dfa-8f3c-44c0e9386611 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:44.516218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2a5e7-77ee-4dfa-8f3c-44c0e9386611 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:44.551405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1ad4a-cdea-4f45-aa8f-ddf03adf63be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:44.552065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1ad4a-cdea-4f45-aa8f-ddf03adf63be None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:44.552454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1ad4a-cdea-4f45-aa8f-ddf03adf63be None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:44.552735 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1ad4a-cdea-4f45-aa8f-ddf03adf63be None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:44.553599 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1ad4a-cdea-4f45-aa8f-ddf03adf63be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:44.553907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1ad4a-cdea-4f45-aa8f-ddf03adf63be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:44.555335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1ad4a-cdea-4f45-aa8f-ddf03adf63be None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:44.572411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 611/1213] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Jul 18 16:33:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:44.578616 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13066e05-3622-4730-9f63-c6eca602e244 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:44.679864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13066e05-3622-4730-9f63-c6eca602e244 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:44.713257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175fae5-ede6-41a8-8b7f-f102d0936580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:44.713932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175fae5-ede6-41a8-8b7f-f102d0936580 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:44.714215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175fae5-ede6-41a8-8b7f-f102d0936580 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:44.714491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175fae5-ede6-41a8-8b7f-f102d0936580 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:44.745572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:44.783995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175fae5-ede6-41a8-8b7f-f102d0936580 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:44.785489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175fae5-ede6-41a8-8b7f-f102d0936580 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:44.885018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 603/1214] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:33:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:33:44.891456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1ad4a-cdea-4f45-aa8f-ddf03adf63be None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:44.910856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:44.992335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1ad4a-cdea-4f45-aa8f-ddf03adf63be None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:45.026365 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c4851-a6d1-486c-9cf6-d45246f79497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:45.027155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c4851-a6d1-486c-9cf6-d45246f79497 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:45.028763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c4851-a6d1-486c-9cf6-d45246f79497 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:45.028763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c4851-a6d1-486c-9cf6-d45246f79497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:45.038885 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c4851-a6d1-486c-9cf6-d45246f79497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:45.040390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c4851-a6d1-486c-9cf6-d45246f79497 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-905c4851-a6d1-486c-9cf6-d45246f79497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:33:45.057866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:45.059325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:45.060278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:33:45.060278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:33:45.060278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:45.060278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:33:45.060278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 612/1215] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:45.065189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175fae5-ede6-41a8-8b7f-f102d0936580 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:45.126819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:45.166651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175fae5-ede6-41a8-8b7f-f102d0936580 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:45.199981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95533c4b-df9c-4b0a-8d17-414dd21e2998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:45.200650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95533c4b-df9c-4b0a-8d17-414dd21e2998 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:45.200965 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95533c4b-df9c-4b0a-8d17-414dd21e2998 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:45.201249 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95533c4b-df9c-4b0a-8d17-414dd21e2998 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:45.202033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95533c4b-df9c-4b0a-8d17-414dd21e2998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:45.202336 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95533c4b-df9c-4b0a-8d17-414dd21e2998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:45.204284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95533c4b-df9c-4b0a-8d17-414dd21e2998 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:45.216014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 604/1216] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:45.221846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c4851-a6d1-486c-9cf6-d45246f79497 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:45.294574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:45.327721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c4851-a6d1-486c-9cf6-d45246f79497 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:45.366083 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c088cff-9dc6-409c-a181-9993940d674e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:45.366867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c088cff-9dc6-409c-a181-9993940d674e None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:45.367145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c088cff-9dc6-409c-a181-9993940d674e None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:45.367416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c088cff-9dc6-409c-a181-9993940d674e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:45.374325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c088cff-9dc6-409c-a181-9993940d674e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:45.376225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c088cff-9dc6-409c-a181-9993940d674e None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5c088cff-9dc6-409c-a181-9993940d674e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:33:45.389827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:45.390933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:45.392049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:33:45.392049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:33:45.392049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:33:45.392049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:33:45.392049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:33:45.392049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:33:45.392049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:33:45.392049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:33:45.392049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 613/1217] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:33:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:45.396902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95533c4b-df9c-4b0a-8d17-414dd21e2998 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:45.495908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95533c4b-df9c-4b0a-8d17-414dd21e2998 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:45.517956 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:45.529805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec55a99-4c48-451b-b129-ad1933a33b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:45.530479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55a99-4c48-451b-b129-ad1933a33b73 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:45.530771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55a99-4c48-451b-b129-ad1933a33b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:45.531054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec55a99-4c48-451b-b129-ad1933a33b73 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:45.532350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec55a99-4c48-451b-b129-ad1933a33b73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:45.532664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec55a99-4c48-451b-b129-ad1933a33b73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:45.534138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec55a99-4c48-451b-b129-ad1933a33b73 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:45.545964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 605/1218] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:33:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:45.553480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c088cff-9dc6-409c-a181-9993940d674e None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:45.656840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c088cff-9dc6-409c-a181-9993940d674e None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:45.685131 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:45.690860 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ca67e-d5d8-4cda-9f7f-aad8b85aabb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:45.691521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ca67e-d5d8-4cda-9f7f-aad8b85aabb7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:45.691806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ca67e-d5d8-4cda-9f7f-aad8b85aabb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:45.692082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ca67e-d5d8-4cda-9f7f-aad8b85aabb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:45.692953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ca67e-d5d8-4cda-9f7f-aad8b85aabb7 None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=22bd4dab1fcd4ad4a8be886646d89242) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:45.693265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ca67e-d5d8-4cda-9f7f-aad8b85aabb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:45.694713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ca67e-d5d8-4cda-9f7f-aad8b85aabb7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:45.700596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 614/1219] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:33:45 2022] GET /identity/v3/role_assignments?user.id=22bd4dab1fcd4ad4a8be886646d89242&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:47.681239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183fd06-09b5-4d2d-8b90-068a625907f5 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:47.681798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183fd06-09b5-4d2d-8b90-068a625907f5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:47.681798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183fd06-09b5-4d2d-8b90-068a625907f5 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:47.683189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 606/1220] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:33:47.688746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50864af4-bc1e-4ea1-a25b-7cd3afd4357d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:47.689231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50864af4-bc1e-4ea1-a25b-7cd3afd4357d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:47.691028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50864af4-bc1e-4ea1-a25b-7cd3afd4357d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:47.709728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-50864af4-bc1e-4ea1-a25b-7cd3afd4357d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:47.710928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-50864af4-bc1e-4ea1-a25b-7cd3afd4357d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:47.779815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 615/1221] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:33:47.804059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183fd06-09b5-4d2d-8b90-068a625907f5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:47.910137 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183fd06-09b5-4d2d-8b90-068a625907f5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:47.946511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09092ca0-8d81-45c0-8316-808984bbdfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:47.947237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09092ca0-8d81-45c0-8316-808984bbdfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:47.947525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09092ca0-8d81-45c0-8316-808984bbdfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:47.947803 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09092ca0-8d81-45c0-8316-808984bbdfc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:47.958323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09092ca0-8d81-45c0-8316-808984bbdfc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-09092ca0-8d81-45c0-8316-808984bbdfc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:33:47.966187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:47.967284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:47.968181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 607/1222] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:33:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:47.973712 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50864af4-bc1e-4ea1-a25b-7cd3afd4357d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:48.079658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50864af4-bc1e-4ea1-a25b-7cd3afd4357d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:48.105245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:48.116729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a962da1-a2bb-4966-8b8b-e378bb6f13c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:48.117432 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962da1-a2bb-4966-8b8b-e378bb6f13c9 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:48.117719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962da1-a2bb-4966-8b8b-e378bb6f13c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:48.117997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962da1-a2bb-4966-8b8b-e378bb6f13c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:48.118747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a962da1-a2bb-4966-8b8b-e378bb6f13c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:48.119055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a962da1-a2bb-4966-8b8b-e378bb6f13c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:48.120432 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a962da1-a2bb-4966-8b8b-e378bb6f13c9 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:48.130207 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 616/1223] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:33:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:48.136163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09092ca0-8d81-45c0-8316-808984bbdfc4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:48.182762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:48.238808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09092ca0-8d81-45c0-8316-808984bbdfc4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:48.271942 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d28ec-3478-4f26-b953-afbbd45e1c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:48.272729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d28ec-3478-4f26-b953-afbbd45e1c3a None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:48.273117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d28ec-3478-4f26-b953-afbbd45e1c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:48.273395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d28ec-3478-4f26-b953-afbbd45e1c3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:48.283653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d28ec-3478-4f26-b953-afbbd45e1c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:48.285162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d28ec-3478-4f26-b953-afbbd45e1c3a None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-d79d28ec-3478-4f26-b953-afbbd45e1c3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:33:48.302351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:48.303606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:48.304781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:33:48.304781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:33:48.304781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:48.304781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:48.304781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 608/1224] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:48.314630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a962da1-a2bb-4966-8b8b-e378bb6f13c9 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:48.419557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a962da1-a2bb-4966-8b8b-e378bb6f13c9 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:48.457196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff94a57-ece2-4e83-83eb-99cb9fbf7d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:48.457864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff94a57-ece2-4e83-83eb-99cb9fbf7d9b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:48.458151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff94a57-ece2-4e83-83eb-99cb9fbf7d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:48.458432 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff94a57-ece2-4e83-83eb-99cb9fbf7d9b None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:48.459157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff94a57-ece2-4e83-83eb-99cb9fbf7d9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:48.459465 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff94a57-ece2-4e83-83eb-99cb9fbf7d9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:48.460915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff94a57-ece2-4e83-83eb-99cb9fbf7d9b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:48.473104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 617/1225] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:48.478922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d28ec-3478-4f26-b953-afbbd45e1c3a None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:48.572701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:48.579929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d28ec-3478-4f26-b953-afbbd45e1c3a None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:48.613799 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81175d92-26c3-42e0-a6a6-c18469c6fce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:48.614851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81175d92-26c3-42e0-a6a6-c18469c6fce0 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:48.615564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81175d92-26c3-42e0-a6a6-c18469c6fce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:48.615564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81175d92-26c3-42e0-a6a6-c18469c6fce0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:48.622389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81175d92-26c3-42e0-a6a6-c18469c6fce0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:48.623812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81175d92-26c3-42e0-a6a6-c18469c6fce0 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-81175d92-26c3-42e0-a6a6-c18469c6fce0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:33:48.639736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:48.648588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:48.649774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:48.649774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:48.649774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:33:48.649774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:33:48.649774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:33:48.649774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:33:48.649774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:33:48.649774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:33:48.649774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:33:48.649774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:48.649774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 609/1226] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:48.650459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff94a57-ece2-4e83-83eb-99cb9fbf7d9b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:48.693653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:48.745464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff94a57-ece2-4e83-83eb-99cb9fbf7d9b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:48.778757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef964f-3df2-470b-a58a-602195cdfdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:48.779423 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef964f-3df2-470b-a58a-602195cdfdfc None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:48.779707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef964f-3df2-470b-a58a-602195cdfdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:48.779984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef964f-3df2-470b-a58a-602195cdfdfc None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:48.780858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef964f-3df2-470b-a58a-602195cdfdfc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:48.781731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef964f-3df2-470b-a58a-602195cdfdfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:48.783175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef964f-3df2-470b-a58a-602195cdfdfc None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:48.800988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 618/1227] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Jul 18 16:33:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:33:48.806969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81175d92-26c3-42e0-a6a6-c18469c6fce0 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:48.908519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81175d92-26c3-42e0-a6a6-c18469c6fce0 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:48.943658 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ab5bb-a2e2-428f-886e-e10793dc798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:48.944546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ab5bb-a2e2-428f-886e-e10793dc798f None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:48.944837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ab5bb-a2e2-428f-886e-e10793dc798f None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:48.945149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ab5bb-a2e2-428f-886e-e10793dc798f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:48.955578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ab5bb-a2e2-428f-886e-e10793dc798f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:48.957049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ab5bb-a2e2-428f-886e-e10793dc798f None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b53ab5bb-a2e2-428f-886e-e10793dc798f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:33:48.975013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:48.976126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:48.982271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:33:48.982271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:33:48.982271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:48.982271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:48.982271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 610/1228] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:48.982663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef964f-3df2-470b-a58a-602195cdfdfc None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:49.081688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef964f-3df2-470b-a58a-602195cdfdfc None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:49.083942 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:49.116931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51525998-3d78-4e36-add9-5342d7fd78ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:49.117588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525998-3d78-4e36-add9-5342d7fd78ff None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:49.117873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525998-3d78-4e36-add9-5342d7fd78ff None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:49.118220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525998-3d78-4e36-add9-5342d7fd78ff None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:49.118870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51525998-3d78-4e36-add9-5342d7fd78ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:49.119170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51525998-3d78-4e36-add9-5342d7fd78ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:49.120544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51525998-3d78-4e36-add9-5342d7fd78ff None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:49.132272 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 619/1229] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:33:49.138271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ab5bb-a2e2-428f-886e-e10793dc798f None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:49.148011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:49.238753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ab5bb-a2e2-428f-886e-e10793dc798f None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:49.272450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4e3f2-e1f5-45c8-997f-f05de30f60c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:49.273259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4e3f2-e1f5-45c8-997f-f05de30f60c6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:49.273541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4e3f2-e1f5-45c8-997f-f05de30f60c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:49.273811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4e3f2-e1f5-45c8-997f-f05de30f60c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:49.280464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4e3f2-e1f5-45c8-997f-f05de30f60c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:49.281968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4e3f2-e1f5-45c8-997f-f05de30f60c6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-4dc4e3f2-e1f5-45c8-997f-f05de30f60c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:33:49.295874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:49.297495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:49.298649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:33:49.298649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:33:49.298649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:33:49.298649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:33:49.298649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:33:49.298649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:33:49.298649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:33:49.298649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:49.298649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 611/1230] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:33:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:49.303498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51525998-3d78-4e36-add9-5342d7fd78ff None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:49.404932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51525998-3d78-4e36-add9-5342d7fd78ff None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:49.440596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bad60b-c0b3-41c9-9f4f-ba6f293b93bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:49.441310 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bad60b-c0b3-41c9-9f4f-ba6f293b93bd None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:49.441596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bad60b-c0b3-41c9-9f4f-ba6f293b93bd None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:49.441870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bad60b-c0b3-41c9-9f4f-ba6f293b93bd None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:49.442663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bad60b-c0b3-41c9-9f4f-ba6f293b93bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:49.442974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bad60b-c0b3-41c9-9f4f-ba6f293b93bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:49.444431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bad60b-c0b3-41c9-9f4f-ba6f293b93bd None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:49.457507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 620/1231] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:33:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:49.464209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4e3f2-e1f5-45c8-997f-f05de30f60c6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:49.531968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:49.568047 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4e3f2-e1f5-45c8-997f-f05de30f60c6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:49.602154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f44785-6699-4567-8834-8e10fc5edacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:49.603026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f44785-6699-4567-8834-8e10fc5edacc None admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:49.603305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f44785-6699-4567-8834-8e10fc5edacc None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:49.603581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f44785-6699-4567-8834-8e10fc5edacc None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/22bd4dab1fcd4ad4a8be886646d89242/roles/3fc59073330c4557ad178629386b0852` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:49.617932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f44785-6699-4567-8834-8e10fc5edacc None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=22bd4dab1fcd4ad4a8be886646d89242, role_id=3fc59073330c4557ad178629386b0852)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:49.619366 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f44785-6699-4567-8834-8e10fc5edacc None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:49.838046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 612/1232] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:33:49 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/22bd4dab1fcd4ad4a8be886646d89242/roles/3fc59073330c4557ad178629386b0852 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:33:51.841160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e240a17-f3ba-45be-80d3-3b5ffe620e64 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:51.841733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e240a17-f3ba-45be-80d3-3b5ffe620e64 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:51.841733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e240a17-f3ba-45be-80d3-3b5ffe620e64 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:51.843075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 621/1233] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:33:51.848548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2e444-5a36-4d44-aa48-2a394bc5f62d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:51.848987 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2e444-5a36-4d44-aa48-2a394bc5f62d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:51.849677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2e444-5a36-4d44-aa48-2a394bc5f62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:51.866412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-76f2e444-5a36-4d44-aa48-2a394bc5f62d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:51.867603 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-76f2e444-5a36-4d44-aa48-2a394bc5f62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:51.934062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 613/1234] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:33:51.957704 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e240a17-f3ba-45be-80d3-3b5ffe620e64 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:51.984035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:52.060383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e240a17-f3ba-45be-80d3-3b5ffe620e64 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:52.094452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840c9a4-d72e-477d-b593-ea99527828c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:52.095170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840c9a4-d72e-477d-b593-ea99527828c0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:52.095448 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840c9a4-d72e-477d-b593-ea99527828c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:52.095720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840c9a4-d72e-477d-b593-ea99527828c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:52.106006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840c9a4-d72e-477d-b593-ea99527828c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-d840c9a4-d72e-477d-b593-ea99527828c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:33:52.114415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:52.124072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:33:52.125610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 622/1235] 10.222.0.88 () {56 vars in 1102 bytes} [Mon Jul 18 16:33:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:52.126542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2e444-5a36-4d44-aa48-2a394bc5f62d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:52.167879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:52.221825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2e444-5a36-4d44-aa48-2a394bc5f62d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:52.254862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b8449-b053-42d4-996c-962addcc77c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:52.255529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8449-b053-42d4-996c-962addcc77c9 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:52.255814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8449-b053-42d4-996c-962addcc77c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:52.256092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8449-b053-42d4-996c-962addcc77c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:52.256912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b8449-b053-42d4-996c-962addcc77c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:52.257222 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b8449-b053-42d4-996c-962addcc77c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:52.258580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b8449-b053-42d4-996c-962addcc77c9 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:52.266494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 614/1236] 10.222.0.88 () {56 vars in 1103 bytes} [Mon Jul 18 16:33:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:52.272458 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840c9a4-d72e-477d-b593-ea99527828c0 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:52.357902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:52.374515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840c9a4-d72e-477d-b593-ea99527828c0 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:52.407970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70f039-09b2-4ab4-9cc8-b7ae223f9bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:52.408766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70f039-09b2-4ab4-9cc8-b7ae223f9bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:52.409085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70f039-09b2-4ab4-9cc8-b7ae223f9bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:52.409367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70f039-09b2-4ab4-9cc8-b7ae223f9bf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:52.419914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70f039-09b2-4ab4-9cc8-b7ae223f9bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:52.421413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70f039-09b2-4ab4-9cc8-b7ae223f9bf7 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-ce70f039-09b2-4ab4-9cc8-b7ae223f9bf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:33:52.439563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:52.440768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:52.442103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:33:52.442103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:33:52.442103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:52.442103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:33:52.442103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 623/1237] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:52.446666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b8449-b053-42d4-996c-962addcc77c9 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:52.549542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b8449-b053-42d4-996c-962addcc77c9 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:52.558272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:52.582723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abd49d-19e8-4a13-9b96-ec9e8ac72ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:52.583380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd49d-19e8-4a13-9b96-ec9e8ac72ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:52.583753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd49d-19e8-4a13-9b96-ec9e8ac72ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:52.583931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd49d-19e8-4a13-9b96-ec9e8ac72ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:52.584647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abd49d-19e8-4a13-9b96-ec9e8ac72ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:52.584973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abd49d-19e8-4a13-9b96-ec9e8ac72ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:52.586331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abd49d-19e8-4a13-9b96-ec9e8ac72ca4 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:52.598360 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 615/1238] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:52.603968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70f039-09b2-4ab4-9cc8-b7ae223f9bf7 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:52.706010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70f039-09b2-4ab4-9cc8-b7ae223f9bf7 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:52.738459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2623b-9742-4622-b066-ce6cb0f9d661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:52.739515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2623b-9742-4622-b066-ce6cb0f9d661 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:52.739799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2623b-9742-4622-b066-ce6cb0f9d661 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:52.740079 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2623b-9742-4622-b066-ce6cb0f9d661 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:52.742808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:52.747796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2623b-9742-4622-b066-ce6cb0f9d661 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:52.749261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2623b-9742-4622-b066-ce6cb0f9d661 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-baf2623b-9742-4622-b066-ce6cb0f9d661 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:33:52.763910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:52.765031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:52.766904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:52.766904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:52.766904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:33:52.766904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:33:52.766904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:33:52.766904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:33:52.766904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:33:52.766904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:33:52.766904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:33:52.766904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:33:52.766904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 624/1239] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:52.771677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abd49d-19e8-4a13-9b96-ec9e8ac72ca4 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:52.878722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abd49d-19e8-4a13-9b96-ec9e8ac72ca4 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:52.912779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707b56e-c67d-42ef-a7fb-6a824ac11e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:52.913574 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707b56e-c67d-42ef-a7fb-6a824ac11e31 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:52.913770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707b56e-c67d-42ef-a7fb-6a824ac11e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:52.914049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707b56e-c67d-42ef-a7fb-6a824ac11e31 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:52.914881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707b56e-c67d-42ef-a7fb-6a824ac11e31 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:52.915192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707b56e-c67d-42ef-a7fb-6a824ac11e31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:52.916626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707b56e-c67d-42ef-a7fb-6a824ac11e31 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:52.934500 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 616/1240] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Jul 18 16:33:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:52.940489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2623b-9742-4622-b066-ce6cb0f9d661 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:53.040862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2623b-9742-4622-b066-ce6cb0f9d661 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:53.074422 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b1d33-228f-4167-b10e-913c8803c408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:53.075070 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1d33-228f-4167-b10e-913c8803c408 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:53.075356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1d33-228f-4167-b10e-913c8803c408 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:53.075630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1d33-228f-4167-b10e-913c8803c408 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:53.136821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:53.144558 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b1d33-228f-4167-b10e-913c8803c408 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:53.146052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b1d33-228f-4167-b10e-913c8803c408 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:53.246865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 625/1241] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:33:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 16:33:53.252859 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707b56e-c67d-42ef-a7fb-6a824ac11e31 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:53.280179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:53.352090 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707b56e-c67d-42ef-a7fb-6a824ac11e31 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:53.384325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a993e-f574-4814-a064-3a84b4e97225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:53.385150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a993e-f574-4814-a064-3a84b4e97225 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:53.385432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a993e-f574-4814-a064-3a84b4e97225 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:53.385703 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a993e-f574-4814-a064-3a84b4e97225 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:53.395775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a993e-f574-4814-a064-3a84b4e97225 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:53.397430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818a993e-f574-4814-a064-3a84b4e97225 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-818a993e-f574-4814-a064-3a84b4e97225 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:33:53.414946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:53.416140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:53.417276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:33:53.417276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:33:53.417276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:53.417276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:53.417276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 617/1242] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:53.421973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b1d33-228f-4167-b10e-913c8803c408 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:53.517188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:53.523004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b1d33-228f-4167-b10e-913c8803c408 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:53.557308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb42da3-84f9-4a2a-a29d-b4f743218625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:53.557973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb42da3-84f9-4a2a-a29d-b4f743218625 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:53.558258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb42da3-84f9-4a2a-a29d-b4f743218625 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:53.558536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb42da3-84f9-4a2a-a29d-b4f743218625 None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:53.559293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb42da3-84f9-4a2a-a29d-b4f743218625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:53.559604 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb42da3-84f9-4a2a-a29d-b4f743218625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:53.561001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb42da3-84f9-4a2a-a29d-b4f743218625 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:53.573292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 626/1243] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:53.579626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a993e-f574-4814-a064-3a84b4e97225 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:53.658090 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:53.679954 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a993e-f574-4814-a064-3a84b4e97225 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:53.714590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba25e7-1c98-4dc8-adb8-5e249696f4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:53.715375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba25e7-1c98-4dc8-adb8-5e249696f4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:53.715655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba25e7-1c98-4dc8-adb8-5e249696f4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:53.715928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba25e7-1c98-4dc8-adb8-5e249696f4e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:53.723367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba25e7-1c98-4dc8-adb8-5e249696f4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:53.724840 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba25e7-1c98-4dc8-adb8-5e249696f4e5 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-a4ba25e7-1c98-4dc8-adb8-5e249696f4e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:33:53.739488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:53.740542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:53.741700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:33:53.741700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:33:53.741700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:33:53.741700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:33:53.741700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:33:53.741700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:33:53.741700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:33:53.741700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:53.741700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 618/1244] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:33:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:53.746665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb42da3-84f9-4a2a-a29d-b4f743218625 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:53.848013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb42da3-84f9-4a2a-a29d-b4f743218625 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:53.882183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5ee21-e47a-4f37-bace-359535f4f6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:53.882833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5ee21-e47a-4f37-bace-359535f4f6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:53.883120 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5ee21-e47a-4f37-bace-359535f4f6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:53.883399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5ee21-e47a-4f37-bace-359535f4f6c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:53.884187 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5ee21-e47a-4f37-bace-359535f4f6c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:53.884498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5ee21-e47a-4f37-bace-359535f4f6c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:53.885972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5ee21-e47a-4f37-bace-359535f4f6c0 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:53.898459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 627/1245] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:33:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:33:53.904677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba25e7-1c98-4dc8-adb8-5e249696f4e5 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:54.005609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba25e7-1c98-4dc8-adb8-5e249696f4e5 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:54.041753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df17b10-9eb6-4b37-881d-44f570acc815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:54.042408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df17b10-9eb6-4b37-881d-44f570acc815 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:54.042689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df17b10-9eb6-4b37-881d-44f570acc815 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:54.042964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df17b10-9eb6-4b37-881d-44f570acc815 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:54.043789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df17b10-9eb6-4b37-881d-44f570acc815 None admin] RBAC: Adding query filter params (role.id=3fc59073330c4557ad178629386b0852, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=22bd4dab1fcd4ad4a8be886646d89242) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:54.044096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df17b10-9eb6-4b37-881d-44f570acc815 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:54.045608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df17b10-9eb6-4b37-881d-44f570acc815 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:54.052062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 619/1246] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:33:53 2022] GET /identity/v3/role_assignments?user.id=22bd4dab1fcd4ad4a8be886646d89242&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=3fc59073330c4557ad178629386b0852 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:56.104730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b80d2f-4b8f-462c-92e3-7d54b3b363ac None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:56.105281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b80d2f-4b8f-462c-92e3-7d54b3b363ac None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:56.105373 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b80d2f-4b8f-462c-92e3-7d54b3b363ac None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:56.106696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 628/1247] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:33:56.112013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a22965-3dea-49d2-8394-3636d6d1c170 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:56.112455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a22965-3dea-49d2-8394-3636d6d1c170 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:56.112572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a22965-3dea-49d2-8394-3636d6d1c170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:56.117939 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:56.129787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c4a22965-3dea-49d2-8394-3636d6d1c170 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:33:56.130977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c4a22965-3dea-49d2-8394-3636d6d1c170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:33:56.197253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 620/1248] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:33:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:33:56.222847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b80d2f-4b8f-462c-92e3-7d54b3b363ac None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:56.242028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:56.327004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b80d2f-4b8f-462c-92e3-7d54b3b363ac None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:56.361746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bb3e2-fc44-48ce-9463-d4cc61ae3e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:56.362532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bb3e2-fc44-48ce-9463-d4cc61ae3e0f None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:56.362816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bb3e2-fc44-48ce-9463-d4cc61ae3e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:56.363087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bb3e2-fc44-48ce-9463-d4cc61ae3e0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:56.373991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bb3e2-fc44-48ce-9463-d4cc61ae3e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-700bb3e2-fc44-48ce-9463-d4cc61ae3e0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:33:56.382127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:56.383058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:33:56.383901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 629/1249] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:33:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:56.389740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a22965-3dea-49d2-8394-3636d6d1c170 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:56.492782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a22965-3dea-49d2-8394-3636d6d1c170 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:56.521676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:56.526755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454df63a-8bf9-4322-bef3-b906d59c9d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:56.527423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454df63a-8bf9-4322-bef3-b906d59c9d02 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:56.527709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454df63a-8bf9-4322-bef3-b906d59c9d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:56.527988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454df63a-8bf9-4322-bef3-b906d59c9d02 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:56.528734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454df63a-8bf9-4322-bef3-b906d59c9d02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:56.529142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454df63a-8bf9-4322-bef3-b906d59c9d02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:56.530523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454df63a-8bf9-4322-bef3-b906d59c9d02 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:56.539133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 621/1250] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:33:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:56.544995 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bb3e2-fc44-48ce-9463-d4cc61ae3e0f None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:56.622309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:56.644179 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bb3e2-fc44-48ce-9463-d4cc61ae3e0f None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:56.677942 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418056b2-eed4-4fb5-a048-fdf02d03b011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:56.678726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418056b2-eed4-4fb5-a048-fdf02d03b011 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:56.679008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418056b2-eed4-4fb5-a048-fdf02d03b011 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:56.679285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418056b2-eed4-4fb5-a048-fdf02d03b011 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:56.689941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418056b2-eed4-4fb5-a048-fdf02d03b011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:56.691393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418056b2-eed4-4fb5-a048-fdf02d03b011 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-418056b2-eed4-4fb5-a048-fdf02d03b011 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:33:56.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:56.715579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:56.716672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:33:56.716672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:33:56.716672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:56.716672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:33:56.716672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 630/1251] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:56.717035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454df63a-8bf9-4322-bef3-b906d59c9d02 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:56.816456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454df63a-8bf9-4322-bef3-b906d59c9d02 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:56.853810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb8d73-385e-413a-8042-1f51aa788096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:56.854475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb8d73-385e-413a-8042-1f51aa788096 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:56.854760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb8d73-385e-413a-8042-1f51aa788096 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:56.855042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb8d73-385e-413a-8042-1f51aa788096 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:56.855797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb8d73-385e-413a-8042-1f51aa788096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:56.856075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb8d73-385e-413a-8042-1f51aa788096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:56.857495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb8d73-385e-413a-8042-1f51aa788096 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:56.870272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 622/1252] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:56.875994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418056b2-eed4-4fb5-a048-fdf02d03b011 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:56.979247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418056b2-eed4-4fb5-a048-fdf02d03b011 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:57.004668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:57.012978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95341ec-ad3e-4173-9191-96e81100ef7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:57.013779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95341ec-ad3e-4173-9191-96e81100ef7b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:57.014058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95341ec-ad3e-4173-9191-96e81100ef7b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:57.014326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95341ec-ad3e-4173-9191-96e81100ef7b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:57.022827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95341ec-ad3e-4173-9191-96e81100ef7b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:57.024288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95341ec-ad3e-4173-9191-96e81100ef7b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-a95341ec-ad3e-4173-9191-96e81100ef7b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:33:57.039814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:57.040952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:57.042119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:57.042119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:57.042119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:33:57.042119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:33:57.042119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:33:57.042119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:33:57.042119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:33:57.042119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:33:57.042119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:33:57.042119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:33:57.042119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 631/1253] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:57.047258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb8d73-385e-413a-8042-1f51aa788096 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:57.104558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:57.149821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb8d73-385e-413a-8042-1f51aa788096 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:57.183729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15053097-de52-4b81-9a4c-c68e9d9df122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:57.184399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15053097-de52-4b81-9a4c-c68e9d9df122 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:57.184684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15053097-de52-4b81-9a4c-c68e9d9df122 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:57.184993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15053097-de52-4b81-9a4c-c68e9d9df122 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:57.185834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15053097-de52-4b81-9a4c-c68e9d9df122 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:57.186144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15053097-de52-4b81-9a4c-c68e9d9df122 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:57.188193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15053097-de52-4b81-9a4c-c68e9d9df122 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:57.205830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 623/1254] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Jul 18 16:33:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:57.214766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95341ec-ad3e-4173-9191-96e81100ef7b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:57.314808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95341ec-ad3e-4173-9191-96e81100ef7b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:57.348718 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6add3-719e-4755-8a40-4e46e2ab9e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:57.349384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6add3-719e-4755-8a40-4e46e2ab9e56 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:57.349704 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6add3-719e-4755-8a40-4e46e2ab9e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:57.353618 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6add3-719e-4755-8a40-4e46e2ab9e56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:57.406707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:57.418817 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6add3-719e-4755-8a40-4e46e2ab9e56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:57.420298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6add3-719e-4755-8a40-4e46e2ab9e56 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:57.523422 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 632/1255] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:33:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:33:57.529307 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15053097-de52-4b81-9a4c-c68e9d9df122 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:57.627227 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15053097-de52-4b81-9a4c-c68e9d9df122 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:57.640832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:57.662706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeeea2-9667-43a5-9353-54003d7c6468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:57.662706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeeea2-9667-43a5-9353-54003d7c6468 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:57.662706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeeea2-9667-43a5-9353-54003d7c6468 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:57.662706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeeea2-9667-43a5-9353-54003d7c6468 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:57.673194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeeea2-9667-43a5-9353-54003d7c6468 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:57.674635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeeea2-9667-43a5-9353-54003d7c6468 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-abbeeea2-9667-43a5-9353-54003d7c6468 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:33:57.692303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:57.693481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:57.694790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:33:57.694790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:33:57.694790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:33:57.694790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:57.694790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 624/1256] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:33:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:57.699354 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6add3-719e-4755-8a40-4e46e2ab9e56 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:57.786815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:57.798157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6add3-719e-4755-8a40-4e46e2ab9e56 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:57.832130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0334c-06aa-4a5f-8fcc-1aa9740dcf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:57.832782 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0334c-06aa-4a5f-8fcc-1aa9740dcf6c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:57.833091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0334c-06aa-4a5f-8fcc-1aa9740dcf6c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:57.833364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0334c-06aa-4a5f-8fcc-1aa9740dcf6c None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:57.834076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0334c-06aa-4a5f-8fcc-1aa9740dcf6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:57.834371 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0334c-06aa-4a5f-8fcc-1aa9740dcf6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:57.835730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0334c-06aa-4a5f-8fcc-1aa9740dcf6c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:57.847481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 633/1257] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:33:57.853072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeeea2-9667-43a5-9353-54003d7c6468 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:57.955594 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeeea2-9667-43a5-9353-54003d7c6468 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:57.988717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5274dc-65d0-4c42-adfa-f0ab2581fc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:57.989813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5274dc-65d0-4c42-adfa-f0ab2581fc11 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:57.990099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5274dc-65d0-4c42-adfa-f0ab2581fc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:57.990373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5274dc-65d0-4c42-adfa-f0ab2581fc11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:57.997464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5274dc-65d0-4c42-adfa-f0ab2581fc11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:57.998513 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5274dc-65d0-4c42-adfa-f0ab2581fc11 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-cd5274dc-65d0-4c42-adfa-f0ab2581fc11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:33:58.013054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:33:58.014219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:33:58.016048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:33:58.016048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:33:58.016048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:33:58.016048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:33:58.016048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:33:58.016048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:33:58.016048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:33:58.016048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:33:58.016048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 625/1258] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:33:58.020744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0334c-06aa-4a5f-8fcc-1aa9740dcf6c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:58.119868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0334c-06aa-4a5f-8fcc-1aa9740dcf6c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:58.153711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2287b-1a16-4b82-ac0a-fd25dbcf8d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:58.154363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2287b-1a16-4b82-ac0a-fd25dbcf8d3d None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:58.154662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2287b-1a16-4b82-ac0a-fd25dbcf8d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:58.154897 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2287b-1a16-4b82-ac0a-fd25dbcf8d3d None admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:58.155657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2287b-1a16-4b82-ac0a-fd25dbcf8d3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:58.155987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2287b-1a16-4b82-ac0a-fd25dbcf8d3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:58.157457 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2287b-1a16-4b82-ac0a-fd25dbcf8d3d None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:58.169290 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 634/1259] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:33:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:33:58.175579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5274dc-65d0-4c42-adfa-f0ab2581fc11 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:33:58.203470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:33:58.277969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5274dc-65d0-4c42-adfa-f0ab2581fc11 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:33:58.311607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc96eb-de5f-4582-8471-5a7606345b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:33:58.312821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc96eb-de5f-4582-8471-5a7606345b9c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:33:58.313137 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc96eb-de5f-4582-8471-5a7606345b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:33:58.321133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc96eb-de5f-4582-8471-5a7606345b9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:33:58.321133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc96eb-de5f-4582-8471-5a7606345b9c None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=22bd4dab1fcd4ad4a8be886646d89242) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:33:58.321133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc96eb-de5f-4582-8471-5a7606345b9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:33:58.321133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc96eb-de5f-4582-8471-5a7606345b9c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:33:58.321499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 626/1260] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:33:58 2022] GET /identity/v3/role_assignments?user.id=22bd4dab1fcd4ad4a8be886646d89242&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=81457691f3a2456fa33358cb6a922d05 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:00.314267 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71d6e3-8687-4189-80bc-4150fcf1b2d5 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:00.314886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71d6e3-8687-4189-80bc-4150fcf1b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:00.314886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be71d6e3-8687-4189-80bc-4150fcf1b2d5 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:00.316144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 635/1261] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:34:00.321550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec432c39-c783-41ec-a759-f27ece40c7e0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:00.322004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec432c39-c783-41ec-a759-f27ece40c7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:00.322096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec432c39-c783-41ec-a759-f27ece40c7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:00.338981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ec432c39-c783-41ec-a759-f27ece40c7e0 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:34:00.340161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ec432c39-c783-41ec-a759-f27ece40c7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:34:00.407523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 627/1262] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:34:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:34:00.433612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be71d6e3-8687-4189-80bc-4150fcf1b2d5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:00.444988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:00.538609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be71d6e3-8687-4189-80bc-4150fcf1b2d5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:00.573285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a046a9-7e3c-4bfd-b008-1f8ea2622ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:00.574006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a046a9-7e3c-4bfd-b008-1f8ea2622ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:00.574293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a046a9-7e3c-4bfd-b008-1f8ea2622ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:00.574568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a046a9-7e3c-4bfd-b008-1f8ea2622ed1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:00.585306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a046a9-7e3c-4bfd-b008-1f8ea2622ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-04a046a9-7e3c-4bfd-b008-1f8ea2622ed1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:34:00.594888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:00.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:34:00.604955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 636/1263] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:34:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 16:34:00.605913 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec432c39-c783-41ec-a759-f27ece40c7e0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:00.622159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:00.703413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec432c39-c783-41ec-a759-f27ece40c7e0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:00.738566 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b807430-bd9d-4e15-ac08-098bc4928190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:00.739477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b807430-bd9d-4e15-ac08-098bc4928190 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:00.739758 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b807430-bd9d-4e15-ac08-098bc4928190 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:00.740029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b807430-bd9d-4e15-ac08-098bc4928190 None admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:00.740768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b807430-bd9d-4e15-ac08-098bc4928190 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:00.741101 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b807430-bd9d-4e15-ac08-098bc4928190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:00.742471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b807430-bd9d-4e15-ac08-098bc4928190 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:00.751442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 628/1264] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:34:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:34:00.757300 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a046a9-7e3c-4bfd-b008-1f8ea2622ed1 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:00.824373 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:00.856222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a046a9-7e3c-4bfd-b008-1f8ea2622ed1 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:00.898540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f79c1-2552-4f8e-843c-8203a05dd2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:00.899340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f79c1-2552-4f8e-843c-8203a05dd2ed None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:00.899661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f79c1-2552-4f8e-843c-8203a05dd2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:00.899940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f79c1-2552-4f8e-843c-8203a05dd2ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:00.911503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f79c1-2552-4f8e-843c-8203a05dd2ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:00.912990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f79c1-2552-4f8e-843c-8203a05dd2ed None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-6a1f79c1-2552-4f8e-843c-8203a05dd2ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:34:00.930626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:00.935570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:00.936662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:34:00.936662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:34:00.936662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:34:00.936662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:34:00.936662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 637/1265] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:00.938423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b807430-bd9d-4e15-ac08-098bc4928190 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:01.021620 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:01.040720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b807430-bd9d-4e15-ac08-098bc4928190 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:01.074193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc483195-ad1c-41f6-8250-498627869484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:01.074894 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc483195-ad1c-41f6-8250-498627869484 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:01.075127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc483195-ad1c-41f6-8250-498627869484 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:01.075405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc483195-ad1c-41f6-8250-498627869484 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:01.076124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc483195-ad1c-41f6-8250-498627869484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:01.076424 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc483195-ad1c-41f6-8250-498627869484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:01.077819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc483195-ad1c-41f6-8250-498627869484 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:01.088834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 629/1266] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:01.095022 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f79c1-2552-4f8e-843c-8203a05dd2ed None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:01.200514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f79c1-2552-4f8e-843c-8203a05dd2ed None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:01.221703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:01.236155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00c088-af4d-4126-af7e-736e38dd6007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:01.236985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00c088-af4d-4126-af7e-736e38dd6007 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:01.237269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00c088-af4d-4126-af7e-736e38dd6007 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:01.237539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00c088-af4d-4126-af7e-736e38dd6007 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:01.245461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00c088-af4d-4126-af7e-736e38dd6007 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:01.246903 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00c088-af4d-4126-af7e-736e38dd6007 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9c00c088-af4d-4126-af7e-736e38dd6007 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:34:01.261091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:01.262263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:01.263509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:01.263509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:01.263509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:34:01.263509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:34:01.263509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:34:01.263509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:34:01.263509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:34:01.263509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:34:01.263509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:34:01.263509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:34:01.263509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 638/1267] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:34:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:01.268364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc483195-ad1c-41f6-8250-498627869484 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:01.370874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc483195-ad1c-41f6-8250-498627869484 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:01.404451 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79543989-1e15-4c25-be5e-6602970d9046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:01.405146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79543989-1e15-4c25-be5e-6602970d9046 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:01.405431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79543989-1e15-4c25-be5e-6602970d9046 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:01.405707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79543989-1e15-4c25-be5e-6602970d9046 None admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:01.407346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79543989-1e15-4c25-be5e-6602970d9046 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:01.407651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79543989-1e15-4c25-be5e-6602970d9046 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:01.409135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79543989-1e15-4c25-be5e-6602970d9046 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:01.418405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:01.427136 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 630/1268] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Jul 18 16:34:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:01.432893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00c088-af4d-4126-af7e-736e38dd6007 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:01.535508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00c088-af4d-4126-af7e-736e38dd6007 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:01.571507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53cbbf-1b71-4585-a6b3-c48f357d4449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:01.572298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53cbbf-1b71-4585-a6b3-c48f357d4449 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:01.572581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53cbbf-1b71-4585-a6b3-c48f357d4449 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:01.572858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53cbbf-1b71-4585-a6b3-c48f357d4449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:01.583786 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53cbbf-1b71-4585-a6b3-c48f357d4449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:01.585277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53cbbf-1b71-4585-a6b3-c48f357d4449 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-3b53cbbf-1b71-4585-a6b3-c48f357d4449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:34:01.602594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:01.603656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:01.604662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:34:01.604662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:34:01.604662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:34:01.604662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:34:01.604662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 639/1269] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:01.610361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79543989-1e15-4c25-be5e-6602970d9046 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:01.711522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79543989-1e15-4c25-be5e-6602970d9046 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:01.746810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a69672-8a86-475c-8d4b-5b45b442ff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:01.747851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69672-8a86-475c-8d4b-5b45b442ff00 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:01.747976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69672-8a86-475c-8d4b-5b45b442ff00 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:01.748226 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69672-8a86-475c-8d4b-5b45b442ff00 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:01.748983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a69672-8a86-475c-8d4b-5b45b442ff00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:01.749403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a69672-8a86-475c-8d4b-5b45b442ff00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:01.751489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a69672-8a86-475c-8d4b-5b45b442ff00 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:01.763285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 631/1270] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:01.769234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53cbbf-1b71-4585-a6b3-c48f357d4449 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:01.801786 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:01.872892 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53cbbf-1b71-4585-a6b3-c48f357d4449 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:01.907471 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52bdfb-b318-413d-a313-169fb694ab09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:01.908265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52bdfb-b318-413d-a313-169fb694ab09 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:01.908548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52bdfb-b318-413d-a313-169fb694ab09 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:01.908824 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52bdfb-b318-413d-a313-169fb694ab09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:01.915480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52bdfb-b318-413d-a313-169fb694ab09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:01.916969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52bdfb-b318-413d-a313-169fb694ab09 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-5b52bdfb-b318-413d-a313-169fb694ab09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:34:01.931998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:01.933225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:01.934501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:34:01.934501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:34:01.934501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:34:01.934501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:34:01.934501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:34:01.934501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:34:01.934501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:34:01.934501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:34:01.934501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 640/1271] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:34:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:01.939861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a69672-8a86-475c-8d4b-5b45b442ff00 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:02.000374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:02.041773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a69672-8a86-475c-8d4b-5b45b442ff00 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:02.076062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df1add-a314-432c-a47a-d1ac474ba301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:02.076716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df1add-a314-432c-a47a-d1ac474ba301 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:02.077035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df1add-a314-432c-a47a-d1ac474ba301 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:02.077313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df1add-a314-432c-a47a-d1ac474ba301 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:02.078101 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df1add-a314-432c-a47a-d1ac474ba301 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:02.078408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df1add-a314-432c-a47a-d1ac474ba301 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:02.079875 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df1add-a314-432c-a47a-d1ac474ba301 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:02.092598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 632/1272] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:34:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:02.098699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52bdfb-b318-413d-a313-169fb694ab09 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:02.191584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:02.200774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52bdfb-b318-413d-a313-169fb694ab09 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:02.234982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247c35b-24de-4b2e-99f5-74260760aaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:02.235920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247c35b-24de-4b2e-99f5-74260760aaef None admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:02.236204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247c35b-24de-4b2e-99f5-74260760aaef None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:02.236483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247c35b-24de-4b2e-99f5-74260760aaef None admin] PATH_INFO: `/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/22bd4dab1fcd4ad4a8be886646d89242/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:02.238550 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247c35b-24de-4b2e-99f5-74260760aaef None admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d156472e4d6a852c592e579f2f39, user_id=22bd4dab1fcd4ad4a8be886646d89242, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:02.240247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247c35b-24de-4b2e-99f5-74260760aaef None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:02.374794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 641/1273] 10.222.0.88 () {58 vars in 1439 bytes} [Mon Jul 18 16:34:02 2022] PUT /identity/v3/projects/7ec2d156472e4d6a852c592e579f2f39/users/22bd4dab1fcd4ad4a8be886646d89242/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:34:04.395584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d980ff-3db8-4568-93db-addcb0b7e71f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:04.396285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d980ff-3db8-4568-93db-addcb0b7e71f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:04.396285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d980ff-3db8-4568-93db-addcb0b7e71f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:04.397553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 633/1274] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:34:04.403090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ecff6-521c-492c-89a7-2e8645441221 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:04.405311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ecff6-521c-492c-89a7-2e8645441221 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:04.405311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ecff6-521c-492c-89a7-2e8645441221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:04.420300 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c61ecff6-521c-492c-89a7-2e8645441221 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:34:04.421569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c61ecff6-521c-492c-89a7-2e8645441221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:34:04.489918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 642/1275] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:34:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 16:34:04.513341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d980ff-3db8-4568-93db-addcb0b7e71f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:04.617848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d980ff-3db8-4568-93db-addcb0b7e71f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:04.636046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:04.651893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90471-5d0c-46f4-8803-b9fe78724380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:04.652630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90471-5d0c-46f4-8803-b9fe78724380 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:04.652935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90471-5d0c-46f4-8803-b9fe78724380 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:04.653213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90471-5d0c-46f4-8803-b9fe78724380 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:04.663175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a90471-5d0c-46f4-8803-b9fe78724380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-35a90471-5d0c-46f4-8803-b9fe78724380 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 16:34:04.670829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:04.676609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:04.677757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 634/1276] 10.222.0.88 () {56 vars in 1096 bytes} [Mon Jul 18 16:34:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:04.678867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ecff6-521c-492c-89a7-2e8645441221 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:04.762211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:04.782528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ecff6-521c-492c-89a7-2e8645441221 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:04.816516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1ed54-e501-4b66-88b1-974797edc256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:04.817210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1ed54-e501-4b66-88b1-974797edc256 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:04.817489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1ed54-e501-4b66-88b1-974797edc256 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:04.817759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1ed54-e501-4b66-88b1-974797edc256 None admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:04.818498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1ed54-e501-4b66-88b1-974797edc256 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:04.818800 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1ed54-e501-4b66-88b1-974797edc256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:04.820159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1ed54-e501-4b66-88b1-974797edc256 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:04.828166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 643/1277] 10.222.0.88 () {56 vars in 1099 bytes} [Mon Jul 18 16:34:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:34:04.833964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90471-5d0c-46f4-8803-b9fe78724380 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:04.938572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90471-5d0c-46f4-8803-b9fe78724380 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:04.975961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d756469-d4ad-4f92-9fdc-e9e974100bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:04.977418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d756469-d4ad-4f92-9fdc-e9e974100bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:04.977707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d756469-d4ad-4f92-9fdc-e9e974100bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:04.977990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d756469-d4ad-4f92-9fdc-e9e974100bb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:04.989960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d756469-d4ad-4f92-9fdc-e9e974100bb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:04.991412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d756469-d4ad-4f92-9fdc-e9e974100bb7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-6d756469-d4ad-4f92-9fdc-e9e974100bb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:34:05.010027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:05.026134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:05.027285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:34:05.027285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:34:05.027285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:34:05.027285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:05.027285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 635/1278] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:05.027657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1ed54-e501-4b66-88b1-974797edc256 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:05.121548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1ed54-e501-4b66-88b1-974797edc256 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:05.156705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27165c07-3823-4e42-9e52-44c1bbc7f53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:05.157523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27165c07-3823-4e42-9e52-44c1bbc7f53c None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:05.158062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27165c07-3823-4e42-9e52-44c1bbc7f53c None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:05.158062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27165c07-3823-4e42-9e52-44c1bbc7f53c None admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:05.158657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27165c07-3823-4e42-9e52-44c1bbc7f53c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:05.158963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27165c07-3823-4e42-9e52-44c1bbc7f53c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:05.160338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27165c07-3823-4e42-9e52-44c1bbc7f53c None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:05.172613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 644/1279] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:05.178342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d756469-d4ad-4f92-9fdc-e9e974100bb7 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:05.211374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:05.280895 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d756469-d4ad-4f92-9fdc-e9e974100bb7 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:05.322200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccd341-407a-4654-8916-8deba8f89226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:05.323005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccd341-407a-4654-8916-8deba8f89226 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:05.323290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccd341-407a-4654-8916-8deba8f89226 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:05.323563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccd341-407a-4654-8916-8deba8f89226 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:05.332045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccd341-407a-4654-8916-8deba8f89226 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:05.333676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccd341-407a-4654-8916-8deba8f89226 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-86ccd341-407a-4654-8916-8deba8f89226 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 16:34:05.351739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:05.353010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:05.354445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:05.354445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:05.354445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 16:34:05.354445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 16:34:05.354445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 16:34:05.354445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 16:34:05.354445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 16:34:05.354445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 16:34:05.354445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 16:34:05.354445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:05.354445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 636/1280] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:34:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:05.358832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27165c07-3823-4e42-9e52-44c1bbc7f53c None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:05.362776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:05.460867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27165c07-3823-4e42-9e52-44c1bbc7f53c None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:05.495051 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78029c-2926-404a-a7f3-f814c262b4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:05.495651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78029c-2926-404a-a7f3-f814c262b4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:05.495935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78029c-2926-404a-a7f3-f814c262b4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:05.496210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78029c-2926-404a-a7f3-f814c262b4a3 None admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:05.497082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78029c-2926-404a-a7f3-f814c262b4a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:05.497393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78029c-2926-404a-a7f3-f814c262b4a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:05.498839 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78029c-2926-404a-a7f3-f814c262b4a3 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:05.517515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 645/1281] 10.222.0.88 () {56 vars in 1143 bytes} [Mon Jul 18 16:34:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:05.523332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccd341-407a-4654-8916-8deba8f89226 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:05.614309 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:05.623527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccd341-407a-4654-8916-8deba8f89226 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:05.657349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2a4e6-8014-43e6-940d-76d4f482591d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:05.657986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2a4e6-8014-43e6-940d-76d4f482591d None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:05.658268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2a4e6-8014-43e6-940d-76d4f482591d None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:05.658542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2a4e6-8014-43e6-940d-76d4f482591d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:05.723167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2a4e6-8014-43e6-940d-76d4f482591d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:05.724634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2a4e6-8014-43e6-940d-76d4f482591d None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:05.833524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:05.845388 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 637/1282] 10.222.0.88 () {58 vars in 1282 bytes} [Mon Jul 18 16:34:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 16:34:05.851592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78029c-2926-404a-a7f3-f814c262b4a3 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:05.919577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:05.960798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78029c-2926-404a-a7f3-f814c262b4a3 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:05.995898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64285020-37ea-4d0d-b58b-0eb30da4418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:05.996756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64285020-37ea-4d0d-b58b-0eb30da4418a None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:05.997078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64285020-37ea-4d0d-b58b-0eb30da4418a None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:05.997359 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64285020-37ea-4d0d-b58b-0eb30da4418a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:06.007896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64285020-37ea-4d0d-b58b-0eb30da4418a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:06.009375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64285020-37ea-4d0d-b58b-0eb30da4418a None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-64285020-37ea-4d0d-b58b-0eb30da4418a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 16:34:06.026940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:06.027996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:06.029164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 16:34:06.029164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 16:34:06.029164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 16:34:06.029164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:34:06.029164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 646/1283] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:06.034346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2a4e6-8014-43e6-940d-76d4f482591d None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:06.135935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2a4e6-8014-43e6-940d-76d4f482591d None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:06.170151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad21f0-fff2-4c26-914b-8b860e5d9bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:06.170860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad21f0-fff2-4c26-914b-8b860e5d9bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:06.171093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad21f0-fff2-4c26-914b-8b860e5d9bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:06.171369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad21f0-fff2-4c26-914b-8b860e5d9bb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:06.172093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad21f0-fff2-4c26-914b-8b860e5d9bb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:06.172397 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad21f0-fff2-4c26-914b-8b860e5d9bb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:06.173787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad21f0-fff2-4c26-914b-8b860e5d9bb2 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:06.185528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 638/1284] 10.222.0.88 () {56 vars in 1109 bytes} [Mon Jul 18 16:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:06.191894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64285020-37ea-4d0d-b58b-0eb30da4418a None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:06.297797 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64285020-37ea-4d0d-b58b-0eb30da4418a None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:06.314034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:06.334719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d8458-dcf6-4cae-9a15-20edfc453663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:06.335521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d8458-dcf6-4cae-9a15-20edfc453663 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:06.335807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d8458-dcf6-4cae-9a15-20edfc453663 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:06.336082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d8458-dcf6-4cae-9a15-20edfc453663 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:06.342976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d8458-dcf6-4cae-9a15-20edfc453663 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:06.344523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d8458-dcf6-4cae-9a15-20edfc453663 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application [None req-9a8d8458-dcf6-4cae-9a15-20edfc453663 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 16:34:06.359013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:06.360541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:06.361478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 16:34:06.361478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 16:34:06.361478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 16:34:06.361478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 16:34:06.361478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 16:34:06.361478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 16:34:06.361478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 16:34:06.361478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: ERROR keystone.server.flask.application Jul 18 16:34:06.361478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 647/1285] 10.222.0.88 () {56 vars in 1111 bytes} [Mon Jul 18 16:34:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:06.366681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad21f0-fff2-4c26-914b-8b860e5d9bb2 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:06.418577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:06.470505 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad21f0-fff2-4c26-914b-8b860e5d9bb2 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:06.504568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3cb36-5027-4374-8e87-fa4ca3558fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:06.505282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3cb36-5027-4374-8e87-fa4ca3558fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:06.505563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3cb36-5027-4374-8e87-fa4ca3558fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:06.505835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3cb36-5027-4374-8e87-fa4ca3558fc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:06.506610 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3cb36-5027-4374-8e87-fa4ca3558fc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:06.506915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3cb36-5027-4374-8e87-fa4ca3558fc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:06.508333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3cb36-5027-4374-8e87-fa4ca3558fc7 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:06.521470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 639/1286] 10.222.0.88 () {56 vars in 1148 bytes} [Mon Jul 18 16:34:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:06.527726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d8458-dcf6-4cae-9a15-20edfc453663 None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:06.629765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d8458-dcf6-4cae-9a15-20edfc453663 None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:06.662957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f82993-8852-407a-8b6b-f62f44098cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:06.664009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f82993-8852-407a-8b6b-f62f44098cdf None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:06.664009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f82993-8852-407a-8b6b-f62f44098cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:06.664270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f82993-8852-407a-8b6b-f62f44098cdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:06.665237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f82993-8852-407a-8b6b-f62f44098cdf None admin] RBAC: Adding query filter params (role.id=81457691f3a2456fa33358cb6a922d05, scope.project.id=7ec2d156472e4d6a852c592e579f2f39, user.id=22bd4dab1fcd4ad4a8be886646d89242) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:06.665551 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f82993-8852-407a-8b6b-f62f44098cdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:06.666996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f82993-8852-407a-8b6b-f62f44098cdf None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:06.673834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 648/1287] 10.222.0.88 () {56 vars in 1374 bytes} [Mon Jul 18 16:34:06 2022] GET /identity/v3/role_assignments?user.id=22bd4dab1fcd4ad4a8be886646d89242&scope.project.id=7ec2d156472e4d6a852c592e579f2f39&role.id=81457691f3a2456fa33358cb6a922d05 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:08.711770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34685d-2fc2-47d3-a315-8a607b1494fb None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:08.712300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34685d-2fc2-47d3-a315-8a607b1494fb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:08.712417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34685d-2fc2-47d3-a315-8a607b1494fb None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:08.713747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 640/1288] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:34:08.719353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3154b-48e3-42bd-a0b3-e7e8ef670917 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:08.719787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3154b-48e3-42bd-a0b3-e7e8ef670917 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:08.719910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3154b-48e3-42bd-a0b3-e7e8ef670917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:08.737196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-76b3154b-48e3-42bd-a0b3-e7e8ef670917 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:34:08.738472 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-76b3154b-48e3-42bd-a0b3-e7e8ef670917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:34:08.768043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:08.806116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 649/1289] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:34:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:34:08.833935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34685d-2fc2-47d3-a315-8a607b1494fb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:08.940976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34685d-2fc2-47d3-a315-8a607b1494fb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:08.952383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:08.976087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086c2c2-f917-4535-8194-ddbca2d279a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:08.976821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086c2c2-f917-4535-8194-ddbca2d279a6 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:08.977151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086c2c2-f917-4535-8194-ddbca2d279a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:08.977427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086c2c2-f917-4535-8194-ddbca2d279a6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:08.978075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086c2c2-f917-4535-8194-ddbca2d279a6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:08.979431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086c2c2-f917-4535-8194-ddbca2d279a6 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-b086c2c2-f917-4535-8194-ddbca2d279a6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:08.991435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:08.992632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:08.993727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 641/1290] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Jul 18 16:34:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:08.998897 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3154b-48e3-42bd-a0b3-e7e8ef670917 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:09.100112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3154b-48e3-42bd-a0b3-e7e8ef670917 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:09.134679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fc6d7-cc21-449a-9781-c5214eef86e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:09.135346 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fc6d7-cc21-449a-9781-c5214eef86e2 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:09.135637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fc6d7-cc21-449a-9781-c5214eef86e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:09.135915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fc6d7-cc21-449a-9781-c5214eef86e2 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:09.136630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fc6d7-cc21-449a-9781-c5214eef86e2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:09.136967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fc6d7-cc21-449a-9781-c5214eef86e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:09.138349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fc6d7-cc21-449a-9781-c5214eef86e2 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:09.143850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 650/1291] 10.222.0.88 () {56 vars in 1116 bytes} [Mon Jul 18 16:34:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:09.149792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086c2c2-f917-4535-8194-ddbca2d279a6 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:09.256295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086c2c2-f917-4535-8194-ddbca2d279a6 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:09.291836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92eeba4-89c1-4b78-bea2-d38364176cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:09.292499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92eeba4-89c1-4b78-bea2-d38364176cbd None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:09.292786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92eeba4-89c1-4b78-bea2-d38364176cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:09.293095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92eeba4-89c1-4b78-bea2-d38364176cbd None admin] PATH_INFO: `/v3/services` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:09.293805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92eeba4-89c1-4b78-bea2-d38364176cbd None admin] RBAC: Adding query filter params (type=placement) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:09.294116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92eeba4-89c1-4b78-bea2-d38364176cbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:09.295492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92eeba4-89c1-4b78-bea2-d38364176cbd None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:09.300104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 642/1292] 10.222.0.88 () {56 vars in 1116 bytes} [Mon Jul 18 16:34:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:11.314851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691fb69-20e5-4b83-814a-9c9021f29587 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:11.315405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691fb69-20e5-4b83-814a-9c9021f29587 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:11.315405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691fb69-20e5-4b83-814a-9c9021f29587 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:11.316807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 651/1293] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:34:11.322080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cb796-77b2-4e0b-91d5-c9d8896cf4ad None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:11.322487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cb796-77b2-4e0b-91d5-c9d8896cf4ad None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:11.322645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cb796-77b2-4e0b-91d5-c9d8896cf4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:11.339493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7c8cb796-77b2-4e0b-91d5-c9d8896cf4ad None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:34:11.347134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7c8cb796-77b2-4e0b-91d5-c9d8896cf4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:34:11.369881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:11.409935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 643/1294] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:34:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:34:11.432729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691fb69-20e5-4b83-814a-9c9021f29587 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:11.465348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:11.533948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691fb69-20e5-4b83-814a-9c9021f29587 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:11.568917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fea6b0-df3a-4d32-b8ee-765a3ddc3b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:11.569511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea6b0-df3a-4d32-b8ee-765a3ddc3b8f None admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:11.569796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea6b0-df3a-4d32-b8ee-765a3ddc3b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:11.570075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fea6b0-df3a-4d32-b8ee-765a3ddc3b8f None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:11.570933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fea6b0-df3a-4d32-b8ee-765a3ddc3b8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:11.572316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fea6b0-df3a-4d32-b8ee-765a3ddc3b8f None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:11.642961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 652/1295] 10.222.0.88 () {60 vars in 1141 bytes} [Mon Jul 18 16:34:11 2022] POST /identity/v3/services => generated 246 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:34:13.677743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a525b3-7994-44e7-b6fa-28e85d2979be None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:13.678295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a525b3-7994-44e7-b6fa-28e85d2979be None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:13.678295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a525b3-7994-44e7-b6fa-28e85d2979be None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:13.679737 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 644/1296] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:34:13.685856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174bb71-4ce0-44f6-83bf-78130cfed850 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:13.687164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174bb71-4ce0-44f6-83bf-78130cfed850 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:13.687164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174bb71-4ce0-44f6-83bf-78130cfed850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:13.705744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9174bb71-4ce0-44f6-83bf-78130cfed850 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:34:13.706986 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9174bb71-4ce0-44f6-83bf-78130cfed850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:34:13.790010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 653/1297] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:34:13 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:34:13.814214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a525b3-7994-44e7-b6fa-28e85d2979be None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:13.927222 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a525b3-7994-44e7-b6fa-28e85d2979be None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:13.966718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b3def-44c0-4eaa-a7c3-9be32ba2a7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:13.967525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b3def-44c0-4eaa-a7c3-9be32ba2a7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:13.967807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b3def-44c0-4eaa-a7c3-9be32ba2a7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:13.968609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b3def-44c0-4eaa-a7c3-9be32ba2a7e3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:13.969428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b3def-44c0-4eaa-a7c3-9be32ba2a7e3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:13.970818 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b3def-44c0-4eaa-a7c3-9be32ba2a7e3 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-456b3def-44c0-4eaa-a7c3-9be32ba2a7e3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:13.982807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:13.984622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:13.985490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 645/1298] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Jul 18 16:34:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:13.991258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174bb71-4ce0-44f6-83bf-78130cfed850 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:14.009350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:14.090893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174bb71-4ce0-44f6-83bf-78130cfed850 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:14.125352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c65986-81dd-4050-a60c-61bd815a5a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:14.126010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c65986-81dd-4050-a60c-61bd815a5a1b None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:14.126289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c65986-81dd-4050-a60c-61bd815a5a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:14.126588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c65986-81dd-4050-a60c-61bd815a5a1b None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:14.127286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c65986-81dd-4050-a60c-61bd815a5a1b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:14.127583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c65986-81dd-4050-a60c-61bd815a5a1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:14.128971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c65986-81dd-4050-a60c-61bd815a5a1b None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:14.133989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 654/1299] 10.222.0.88 () {56 vars in 1116 bytes} [Mon Jul 18 16:34:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:14.140533 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b3def-44c0-4eaa-a7c3-9be32ba2a7e3 None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:14.173476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:14.241786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b3def-44c0-4eaa-a7c3-9be32ba2a7e3 None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:14.275408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864b342-037d-481f-a393-3850490ffe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:14.276053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864b342-037d-481f-a393-3850490ffe51 None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:14.276336 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864b342-037d-481f-a393-3850490ffe51 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:14.276610 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864b342-037d-481f-a393-3850490ffe51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:14.277423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864b342-037d-481f-a393-3850490ffe51 None admin] RBAC: Adding query filter params (interface=public, service_id=8d8508465e14452ca6292792fa348a1b, region_id=RegionOne) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:14.284924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864b342-037d-481f-a393-3850490ffe51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:14.284924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864b342-037d-481f-a393-3850490ffe51 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:14.284924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 646/1300] 10.222.0.88 () {56 vars in 1251 bytes} [Mon Jul 18 16:34:14 2022] GET /identity/v3/endpoints?service_id=8d8508465e14452ca6292792fa348a1b&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:14.289999 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c65986-81dd-4050-a60c-61bd815a5a1b None admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:14.379016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:14.397674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c65986-81dd-4050-a60c-61bd815a5a1b None admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:14.431521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8b9d7-f921-4f16-8b62-cafee10f7cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:14.432180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8b9d7-f921-4f16-8b62-cafee10f7cfe None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:14.432473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8b9d7-f921-4f16-8b62-cafee10f7cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:14.432753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8b9d7-f921-4f16-8b62-cafee10f7cfe None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:14.433468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8b9d7-f921-4f16-8b62-cafee10f7cfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:14.434843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8b9d7-f921-4f16-8b62-cafee10f7cfe None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:14.442089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 655/1301] 10.222.0.88 () {56 vars in 1087 bytes} [Mon Jul 18 16:34:14 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:34:16.445891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85345452-4618-4c5d-8fe5-8597c7d3124f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:16.446483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85345452-4618-4c5d-8fe5-8597c7d3124f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:16.446483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85345452-4618-4c5d-8fe5-8597c7d3124f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:16.447996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 647/1302] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:34:16.453763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d103c8-9f61-49a1-9e1a-95d07bbd230f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:16.454164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d103c8-9f61-49a1-9e1a-95d07bbd230f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:16.454326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d103c8-9f61-49a1-9e1a-95d07bbd230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:16.475615 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-02d103c8-9f61-49a1-9e1a-95d07bbd230f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:34:16.476823 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-02d103c8-9f61-49a1-9e1a-95d07bbd230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:34:16.544078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 656/1303] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:34:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 16:34:16.566529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85345452-4618-4c5d-8fe5-8597c7d3124f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:16.662722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:16.669700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85345452-4618-4c5d-8fe5-8597c7d3124f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:16.704276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa2643-4c95-4aaf-8770-92a3fef91b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:16.705027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa2643-4c95-4aaf-8770-92a3fef91b0c None admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:16.705316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa2643-4c95-4aaf-8770-92a3fef91b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:16.705592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa2643-4c95-4aaf-8770-92a3fef91b0c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:16.706238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa2643-4c95-4aaf-8770-92a3fef91b0c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:16.707595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa2643-4c95-4aaf-8770-92a3fef91b0c None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application [None req-bdfa2643-4c95-4aaf-8770-92a3fef91b0c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 16:34:16.719334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 16:34:16.720562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application with Lock( Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self._enter() Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application return self.creator() Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application created_value = creator( Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: ERROR keystone.server.flask.application Jul 18 16:34:16.721914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 648/1304] 10.222.0.88 () {56 vars in 1117 bytes} [Mon Jul 18 16:34:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 16:34:16.726567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d103c8-9f61-49a1-9e1a-95d07bbd230f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:16.803274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:16.828363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d103c8-9f61-49a1-9e1a-95d07bbd230f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:16.864484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1ab72-0f19-487e-935b-fff1e9762975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:16.865182 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1ab72-0f19-487e-935b-fff1e9762975 None admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:16.865470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1ab72-0f19-487e-935b-fff1e9762975 None admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:16.865749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1ab72-0f19-487e-935b-fff1e9762975 None admin] PATH_INFO: `/v3/services` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:16.866462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1ab72-0f19-487e-935b-fff1e9762975 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:34:16.866771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1ab72-0f19-487e-935b-fff1e9762975 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:16.868237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1ab72-0f19-487e-935b-fff1e9762975 None admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:16.873707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 657/1305] 10.222.0.88 () {56 vars in 1116 bytes} [Mon Jul 18 16:34:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:34:16.880069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa2643-4c95-4aaf-8770-92a3fef91b0c None admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:34:16.984287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa2643-4c95-4aaf-8770-92a3fef91b0c None admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:34:17.018977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd15d0d-6807-4744-b7fe-6f43821b7d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:34:17.019679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15d0d-6807-4744-b7fe-6f43821b7d42 None admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:34:17.019966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15d0d-6807-4744-b7fe-6f43821b7d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:34:17.020243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15d0d-6807-4744-b7fe-6f43821b7d42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:34:17.021015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd15d0d-6807-4744-b7fe-6f43821b7d42 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:34:17.022397 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd15d0d-6807-4744-b7fe-6f43821b7d42 None admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:34:17.141145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 649/1306] 10.222.0.88 () {60 vars in 1144 bytes} [Mon Jul 18 16:34:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:34:26.870448 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=30 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:34:27.135381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:10.111600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488da5-31cb-47d1-a82d-ed241d95edf4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:10.122600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488da5-31cb-47d1-a82d-ed241d95edf4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:10.122600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488da5-31cb-47d1-a82d-ed241d95edf4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:10.122813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424527b-c0d3-49d0-ba6b-03bea3884e80 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:10.122813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424527b-c0d3-49d0-ba6b-03bea3884e80 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:10.122813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424527b-c0d3-49d0-ba6b-03bea3884e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:10.123356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 658/1307] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:10.197641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3424527b-c0d3-49d0-ba6b-03bea3884e80 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:10.199365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3424527b-c0d3-49d0-ba6b-03bea3884e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:10.325151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 650/1308] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:41:10.375980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c715168-248c-4e9e-a117-1d204f3539b9 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:10.376268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c715168-248c-4e9e-a117-1d204f3539b9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:10.376467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c715168-248c-4e9e-a117-1d204f3539b9 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:10.378353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 659/1309] 10.222.0.88 () {54 vars in 965 bytes} [Mon Jul 18 16:41:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:10.385658 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03990e6-7eec-4547-80c1-c6c3e2a04efb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:10.386249 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03990e6-7eec-4547-80c1-c6c3e2a04efb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:10.386678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03990e6-7eec-4547-80c1-c6c3e2a04efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:10.489640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c03990e6-7eec-4547-80c1-c6c3e2a04efb None None] MFA Rules not processed for user `de8484c3b114467ea59bebb2460ed546`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:10.490886 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c03990e6-7eec-4547-80c1-c6c3e2a04efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:10.513347 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:10.585563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c715168-248c-4e9e-a117-1d204f3539b9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:10.587692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2c715168-248c-4e9e-a117-1d204f3539b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:10.594754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 651/1310] 10.222.0.88 () {58 vars in 1062 bytes} [Mon Jul 18 16:41:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:10.722976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c715168-248c-4e9e-a117-1d204f3539b9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:10.805493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87aa2f0-bb4f-4da6-b5dc-d7ea61646575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:10.806153 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87aa2f0-bb4f-4da6-b5dc-d7ea61646575 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:10.806436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87aa2f0-bb4f-4da6-b5dc-d7ea61646575 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:10.806711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87aa2f0-bb4f-4da6-b5dc-d7ea61646575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:10.809055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f87aa2f0-bb4f-4da6-b5dc-d7ea61646575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:10.867697 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:10.929415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87aa2f0-bb4f-4da6-b5dc-d7ea61646575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:10.931183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87aa2f0-bb4f-4da6-b5dc-d7ea61646575 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:10.953155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 660/1311] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:41:13.353132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f65cd9-dbd3-4ad8-95a8-6305b8618e50 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:13.353680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f65cd9-dbd3-4ad8-95a8-6305b8618e50 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:13.353680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f65cd9-dbd3-4ad8-95a8-6305b8618e50 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:13.355884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 652/1312] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:13.361635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc5f5-fd57-49bb-b0bb-bd182d48f092 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:13.362194 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc5f5-fd57-49bb-b0bb-bd182d48f092 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:13.362194 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc5f5-fd57-49bb-b0bb-bd182d48f092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:13.451601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6affc5f5-fd57-49bb-b0bb-bd182d48f092 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:13.452826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6affc5f5-fd57-49bb-b0bb-bd182d48f092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:13.552125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 661/1313] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:13.590177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f65cd9-dbd3-4ad8-95a8-6305b8618e50 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:13.591717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-80f65cd9-dbd3-4ad8-95a8-6305b8618e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:13.726673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f65cd9-dbd3-4ad8-95a8-6305b8618e50 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:13.788303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:13.790768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a938c-5c8c-4958-9ffe-6658e62b8970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:13.791965 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a938c-5c8c-4958-9ffe-6658e62b8970 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:13.792248 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a938c-5c8c-4958-9ffe-6658e62b8970 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:13.792527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a938c-5c8c-4958-9ffe-6658e62b8970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:13.794241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-418a938c-5c8c-4958-9ffe-6658e62b8970 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:13.918758 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a938c-5c8c-4958-9ffe-6658e62b8970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:13.920726 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a938c-5c8c-4958-9ffe-6658e62b8970 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:13.944033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 653/1314] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:16.685161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35671f7b-4a3d-4399-8d3e-d0fc5da8dcb8 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.685715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35671f7b-4a3d-4399-8d3e-d0fc5da8dcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.685715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35671f7b-4a3d-4399-8d3e-d0fc5da8dcb8 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.686994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 662/1315] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:16.692285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5c07a-a81a-4baa-83a5-405ba056396b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.692738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5c07a-a81a-4baa-83a5-405ba056396b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.692843 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5c07a-a81a-4baa-83a5-405ba056396b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.758610 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-55a5c07a-a81a-4baa-83a5-405ba056396b None None] MFA Rules not processed for user `72ac367241b34a4bafbea8748bd53095`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:16.759852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-55a5c07a-a81a-4baa-83a5-405ba056396b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:16.811920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:16.856074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 654/1316] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:16.908122 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f13d9e-2e71-47a8-a449-feeb88769394 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.908409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f13d9e-2e71-47a8-a449-feeb88769394 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.908689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f13d9e-2e71-47a8-a449-feeb88769394 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.910095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 663/1317] 10.222.0.88 () {54 vars in 965 bytes} [Mon Jul 18 16:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:16.915983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c2aa4-93e8-43b2-9a85-7f94e82da83e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:16.916813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c2aa4-93e8-43b2-9a85-7f94e82da83e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:16.917164 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c2aa4-93e8-43b2-9a85-7f94e82da83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:16.990677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-888c2aa4-93e8-43b2-9a85-7f94e82da83e None None] MFA Rules not processed for user `de8484c3b114467ea59bebb2460ed546`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:16.991798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-888c2aa4-93e8-43b2-9a85-7f94e82da83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:17.084552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 655/1318] 10.222.0.88 () {58 vars in 1062 bytes} [Mon Jul 18 16:41:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:17.093852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f13d9e-2e71-47a8-a449-feeb88769394 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:17.095428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c2f13d9e-2e71-47a8-a449-feeb88769394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:17.106462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:17.206738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f13d9e-2e71-47a8-a449-feeb88769394 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:17.284483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8020d81-c103-4f00-9606-6d3a1edc4e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:17.285185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8020d81-c103-4f00-9606-6d3a1edc4e68 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:17.285472 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8020d81-c103-4f00-9606-6d3a1edc4e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:17.285751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8020d81-c103-4f00-9606-6d3a1edc4e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:17.287455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d8020d81-c103-4f00-9606-6d3a1edc4e68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:17.353190 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:17.405657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8020d81-c103-4f00-9606-6d3a1edc4e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:17.407118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8020d81-c103-4f00-9606-6d3a1edc4e68 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:17.427168 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 664/1319] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:17 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:20.701932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30abad-be54-44eb-b95c-80619b49e4a4 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:20.702433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30abad-be54-44eb-b95c-80619b49e4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:20.702433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30abad-be54-44eb-b95c-80619b49e4a4 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:20.703797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 656/1320] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:20.709092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2397d41-ca0a-4751-896a-c902b9119b1a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:20.709519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2397d41-ca0a-4751-896a-c902b9119b1a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:20.709658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2397d41-ca0a-4751-896a-c902b9119b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:20.775271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a2397d41-ca0a-4751-896a-c902b9119b1a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:20.777104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a2397d41-ca0a-4751-896a-c902b9119b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:20.873720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 665/1321] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:20.913532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30abad-be54-44eb-b95c-80619b49e4a4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:20.915072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0e30abad-be54-44eb-b95c-80619b49e4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:20.965439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:21.019288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30abad-be54-44eb-b95c-80619b49e4a4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:21.091370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a96cce-42b4-46a6-bc10-6a50acf63896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:21.092003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a96cce-42b4-46a6-bc10-6a50acf63896 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:21.092283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a96cce-42b4-46a6-bc10-6a50acf63896 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:21.092553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a96cce-42b4-46a6-bc10-6a50acf63896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:21.094226 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d4a96cce-42b4-46a6-bc10-6a50acf63896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:21.189437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:21.203223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a96cce-42b4-46a6-bc10-6a50acf63896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:21.213154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a96cce-42b4-46a6-bc10-6a50acf63896 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:21.222508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 657/1322] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:24.269326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9213d6-c69d-476a-b9d9-72a1653b64d0 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:24.269746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9213d6-c69d-476a-b9d9-72a1653b64d0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:24.269962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9213d6-c69d-476a-b9d9-72a1653b64d0 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:24.272931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 666/1323] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:24.277600 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db83d3-d771-4fe0-a4c0-05599f93ddda None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:24.278034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db83d3-d771-4fe0-a4c0-05599f93ddda None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:24.278177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db83d3-d771-4fe0-a4c0-05599f93ddda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:24.343373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d1db83d3-d771-4fe0-a4c0-05599f93ddda None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:24.344575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d1db83d3-d771-4fe0-a4c0-05599f93ddda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:24.449827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 658/1324] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:24.497667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9213d6-c69d-476a-b9d9-72a1653b64d0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:24.499215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ca9213d6-c69d-476a-b9d9-72a1653b64d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:24.526878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:24.623196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9213d6-c69d-476a-b9d9-72a1653b64d0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:24.706611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae57e7-943a-4e91-b3d8-3bac7db5f26a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:24.707271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae57e7-943a-4e91-b3d8-3bac7db5f26a service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:24.707555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae57e7-943a-4e91-b3d8-3bac7db5f26a service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:24.707831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae57e7-943a-4e91-b3d8-3bac7db5f26a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:24.709525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-49ae57e7-943a-4e91-b3d8-3bac7db5f26a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:24.802455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:24.842764 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae57e7-943a-4e91-b3d8-3bac7db5f26a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:24.844242 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae57e7-943a-4e91-b3d8-3bac7db5f26a service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:24.868696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 667/1325] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:27.010718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a32e-4bdc-4059-a088-a2ac225c7aed None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:27.011257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a32e-4bdc-4059-a088-a2ac225c7aed None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:27.011257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a32e-4bdc-4059-a088-a2ac225c7aed None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:27.013222 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 659/1326] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:27.019208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bda147-7900-4a62-ae3a-f934374e1ecd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:27.019757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bda147-7900-4a62-ae3a-f934374e1ecd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:27.019983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bda147-7900-4a62-ae3a-f934374e1ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:27.087447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-03bda147-7900-4a62-ae3a-f934374e1ecd None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:27.088742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-03bda147-7900-4a62-ae3a-f934374e1ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:27.199085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 668/1327] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:27.236364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991a32e-4bdc-4059-a088-a2ac225c7aed None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:27.237966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b991a32e-4bdc-4059-a088-a2ac225c7aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:27.301371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:27.347372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991a32e-4bdc-4059-a088-a2ac225c7aed None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:27.432120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b0ffd-e781-43e5-849a-adc425dfdf72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:27.432784 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b0ffd-e781-43e5-849a-adc425dfdf72 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:27.433106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b0ffd-e781-43e5-849a-adc425dfdf72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:27.433375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b0ffd-e781-43e5-849a-adc425dfdf72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:27.435061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bb4b0ffd-e781-43e5-849a-adc425dfdf72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:27.568788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:27.571340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b0ffd-e781-43e5-849a-adc425dfdf72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:27.573438 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b0ffd-e781-43e5-849a-adc425dfdf72 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:27.591976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 660/1328] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:29.774443 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7554f-cd74-4385-8b1a-eabf683ea18b None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:29.774989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7554f-cd74-4385-8b1a-eabf683ea18b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:29.774989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7554f-cd74-4385-8b1a-eabf683ea18b None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:29.776982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 669/1329] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:29.782490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e0e13-697b-4e29-9ff0-747fb0c7a7a4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:29.787805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e0e13-697b-4e29-9ff0-747fb0c7a7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:29.787805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e0e13-697b-4e29-9ff0-747fb0c7a7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:29.859039 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-588e0e13-697b-4e29-9ff0-747fb0c7a7a4 None None] MFA Rules not processed for user `72ac367241b34a4bafbea8748bd53095`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:29.860248 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-588e0e13-697b-4e29-9ff0-747fb0c7a7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:29.967646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 661/1330] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:30.006504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7554f-cd74-4385-8b1a-eabf683ea18b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:30.008084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e5f7554f-cd74-4385-8b1a-eabf683ea18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:30.040792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:30.120709 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7554f-cd74-4385-8b1a-eabf683ea18b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:30.197906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdf618-6b6d-47da-b9fc-ccb15d440ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:30.198551 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf618-6b6d-47da-b9fc-ccb15d440ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:30.198829 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf618-6b6d-47da-b9fc-ccb15d440ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:30.199102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf618-6b6d-47da-b9fc-ccb15d440ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:30.200783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e0cdf618-6b6d-47da-b9fc-ccb15d440ad7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:30.281634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:30.315008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdf618-6b6d-47da-b9fc-ccb15d440ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:30.316969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdf618-6b6d-47da-b9fc-ccb15d440ad7 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:30.339311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 670/1331] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:30 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:33.062619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c166b-5fa5-4cd1-a418-88e6fe3f912c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:33.063157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c166b-5fa5-4cd1-a418-88e6fe3f912c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:33.063254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c166b-5fa5-4cd1-a418-88e6fe3f912c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:33.064777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 662/1332] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:33.070723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d3f7f-757e-4d04-9c3f-1167d492ce86 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:33.074049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d3f7f-757e-4d04-9c3f-1167d492ce86 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:33.074049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d3f7f-757e-4d04-9c3f-1167d492ce86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:33.157108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-801d3f7f-757e-4d04-9c3f-1167d492ce86 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:33.158373 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-801d3f7f-757e-4d04-9c3f-1167d492ce86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:33.272856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 671/1333] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:33.312685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c166b-5fa5-4cd1-a418-88e6fe3f912c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:33.314454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a05c166b-5fa5-4cd1-a418-88e6fe3f912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:33.404568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:33.446306 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c166b-5fa5-4cd1-a418-88e6fe3f912c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:33.526725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f221c-3fc6-46cb-8baf-9d11ac79a315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:33.527282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f221c-3fc6-46cb-8baf-9d11ac79a315 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:33.527560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f221c-3fc6-46cb-8baf-9d11ac79a315 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:33.527829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f221c-3fc6-46cb-8baf-9d11ac79a315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:33.529541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5a2f221c-3fc6-46cb-8baf-9d11ac79a315 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:33.649547 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f221c-3fc6-46cb-8baf-9d11ac79a315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:33.651082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f221c-3fc6-46cb-8baf-9d11ac79a315 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:33.665285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:33.678714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 663/1334] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:35.772309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed8df8-dd1b-418f-9433-764d90e8134e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:35.772849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed8df8-dd1b-418f-9433-764d90e8134e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:35.772995 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed8df8-dd1b-418f-9433-764d90e8134e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:35.774375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 672/1335] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:35.780300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed136bcb-2717-49d2-b531-a42bc563c3a3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:35.780747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed136bcb-2717-49d2-b531-a42bc563c3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:35.780855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed136bcb-2717-49d2-b531-a42bc563c3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:35.846447 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ed136bcb-2717-49d2-b531-a42bc563c3a3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:35.847695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ed136bcb-2717-49d2-b531-a42bc563c3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:35.944080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 664/1336] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:35.984357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed8df8-dd1b-418f-9433-764d90e8134e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:35.986023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2eed8df8-dd1b-418f-9433-764d90e8134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:36.027262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:36.093089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed8df8-dd1b-418f-9433-764d90e8134e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:36.166418 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31da1e-6021-41dd-82a0-4a1804d11bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:36.167071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31da1e-6021-41dd-82a0-4a1804d11bfc service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:36.167361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31da1e-6021-41dd-82a0-4a1804d11bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:36.167642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31da1e-6021-41dd-82a0-4a1804d11bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:36.169342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fc31da1e-6021-41dd-82a0-4a1804d11bfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:36.252955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:36.278355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31da1e-6021-41dd-82a0-4a1804d11bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:36.279813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31da1e-6021-41dd-82a0-4a1804d11bfc service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:36.298383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 673/1337] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:39.018706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975c7ad-2c18-47e3-922e-1993643b168a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:39.019256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975c7ad-2c18-47e3-922e-1993643b168a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:39.019256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975c7ad-2c18-47e3-922e-1993643b168a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:39.020643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 665/1338] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:39.025819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe77e7-597a-43cb-bc50-22e5a94f88a6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:39.026232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe77e7-597a-43cb-bc50-22e5a94f88a6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:39.026378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe77e7-597a-43cb-bc50-22e5a94f88a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:39.090492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-89fe77e7-597a-43cb-bc50-22e5a94f88a6 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:39.091713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-89fe77e7-597a-43cb-bc50-22e5a94f88a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:39.188108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 674/1339] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:39.224177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975c7ad-2c18-47e3-922e-1993643b168a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:39.226055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4975c7ad-2c18-47e3-922e-1993643b168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:39.306420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:39.336291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975c7ad-2c18-47e3-922e-1993643b168a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:39.418710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa87eee-c704-4e04-b0e2-745edae62e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:39.419360 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa87eee-c704-4e04-b0e2-745edae62e5d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:39.419639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa87eee-c704-4e04-b0e2-745edae62e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:39.419912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa87eee-c704-4e04-b0e2-745edae62e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:39.421651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ffa87eee-c704-4e04-b0e2-745edae62e5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:39.535259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa87eee-c704-4e04-b0e2-745edae62e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:39.536659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa87eee-c704-4e04-b0e2-745edae62e5d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:39.557186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 666/1340] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:44.133578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac763b-fec4-403e-b2af-60343f208dcd None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.134088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac763b-fec4-403e-b2af-60343f208dcd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.134088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac763b-fec4-403e-b2af-60343f208dcd None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.137278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 675/1341] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:44.140823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834444f-4377-434d-812f-1123e6ffaeb1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.141314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834444f-4377-434d-812f-1123e6ffaeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.141568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834444f-4377-434d-812f-1123e6ffaeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.145656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:44.225708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0834444f-4377-434d-812f-1123e6ffaeb1 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:44.226978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0834444f-4377-434d-812f-1123e6ffaeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:44.341080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 667/1342] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:44.380296 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac763b-fec4-403e-b2af-60343f208dcd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:44.381925 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-90ac763b-fec4-403e-b2af-60343f208dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:44.438169 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:44.503835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac763b-fec4-403e-b2af-60343f208dcd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:44.582749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97e723-18aa-4f31-b85b-33f47a858887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:44.583647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97e723-18aa-4f31-b85b-33f47a858887 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:44.583967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97e723-18aa-4f31-b85b-33f47a858887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:44.584250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97e723-18aa-4f31-b85b-33f47a858887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:44.585990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7c97e723-18aa-4f31-b85b-33f47a858887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:44.685727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:44.703806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97e723-18aa-4f31-b85b-33f47a858887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:44.704819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97e723-18aa-4f31-b85b-33f47a858887 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:44.731837 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 676/1343] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:48.049614 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd109617-2075-4385-aa69-4185ac8205d1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:48.050189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd109617-2075-4385-aa69-4185ac8205d1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:48.050189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd109617-2075-4385-aa69-4185ac8205d1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:48.051475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 668/1344] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:48.057812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5573995-5c59-4733-8809-8053ba8baa45 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:48.058378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5573995-5c59-4733-8809-8053ba8baa45 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:48.058477 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5573995-5c59-4733-8809-8053ba8baa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:48.135930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c5573995-5c59-4733-8809-8053ba8baa45 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:48.137225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c5573995-5c59-4733-8809-8053ba8baa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:48.264078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 677/1345] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:48.306452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd109617-2075-4385-aa69-4185ac8205d1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:48.307942 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fd109617-2075-4385-aa69-4185ac8205d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:48.423119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:48.441926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd109617-2075-4385-aa69-4185ac8205d1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:48.527950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24ef77-ce06-478f-8309-a2f562804cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:48.529157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24ef77-ce06-478f-8309-a2f562804cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:48.529770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24ef77-ce06-478f-8309-a2f562804cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:48.530269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24ef77-ce06-478f-8309-a2f562804cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:48.533605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6e24ef77-ce06-478f-8309-a2f562804cf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:48.659720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24ef77-ce06-478f-8309-a2f562804cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:48.661159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24ef77-ce06-478f-8309-a2f562804cf6 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:48.680860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 669/1346] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:52.989757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ba173-ae5f-43bb-a750-7cf3145b3428 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:52.990312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ba173-ae5f-43bb-a750-7cf3145b3428 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:52.990312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ba173-ae5f-43bb-a750-7cf3145b3428 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:52.991679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 678/1347] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:52.997204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec45b5-e64c-44ae-b0f0-35aad481ca2e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:52.997648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec45b5-e64c-44ae-b0f0-35aad481ca2e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:52.997776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec45b5-e64c-44ae-b0f0-35aad481ca2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:53.013626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:53.069196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-98ec45b5-e64c-44ae-b0f0-35aad481ca2e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:53.070359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-98ec45b5-e64c-44ae-b0f0-35aad481ca2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:53.175726 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 670/1348] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:41:53.215671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ba173-ae5f-43bb-a750-7cf3145b3428 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:41:53.217309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-482ba173-ae5f-43bb-a750-7cf3145b3428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:53.284637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:53.333629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ba173-ae5f-43bb-a750-7cf3145b3428 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:41:53.411264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367b374-ff9f-48c9-b4d5-2581b36ba51b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:41:53.411918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367b374-ff9f-48c9-b4d5-2581b36ba51b service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:53.412205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367b374-ff9f-48c9-b4d5-2581b36ba51b service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:53.412485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367b374-ff9f-48c9-b4d5-2581b36ba51b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:53.414175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7367b374-ff9f-48c9-b4d5-2581b36ba51b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:53.523888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:41:53.527553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367b374-ff9f-48c9-b4d5-2581b36ba51b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:41:53.529051 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367b374-ff9f-48c9-b4d5-2581b36ba51b service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:41:53.547588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 679/1349] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:41:58.550362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1e083-b602-4db2-9d87-1922af1b35b9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:58.551203 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1e083-b602-4db2-9d87-1922af1b35b9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:58.551654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1e083-b602-4db2-9d87-1922af1b35b9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:58.553529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 671/1350] 10.222.0.88 () {54 vars in 921 bytes} [Mon Jul 18 16:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:41:58.559155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488e094-b4ee-4df3-bece-9e39eb9828a5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:41:58.559622 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488e094-b4ee-4df3-bece-9e39eb9828a5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:41:58.559717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488e094-b4ee-4df3-bece-9e39eb9828a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:41:58.692571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8488e094-b4ee-4df3-bece-9e39eb9828a5 None None] MFA Rules not processed for user `22bd4dab1fcd4ad4a8be886646d89242`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:41:58.693831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8488e094-b4ee-4df3-bece-9e39eb9828a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:41:58.820266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 680/1351] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Jul 18 16:41:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:02.981686 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d8de0-4ad2-41ea-88ae-40666fbf0cd5 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:02.982209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d8de0-4ad2-41ea-88ae-40666fbf0cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:02.982209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d8de0-4ad2-41ea-88ae-40666fbf0cd5 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:02.983677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 672/1352] 10.222.0.88 () {54 vars in 921 bytes} [Mon Jul 18 16:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:02.989329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb187c-749b-47cd-8929-7061db2469ef None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:02.989790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb187c-749b-47cd-8929-7061db2469ef None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:02.989905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb187c-749b-47cd-8929-7061db2469ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:03.048842 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:03.073728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-01eb187c-749b-47cd-8929-7061db2469ef None None] MFA Rules not processed for user `22bd4dab1fcd4ad4a8be886646d89242`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:03.074945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-01eb187c-749b-47cd-8929-7061db2469ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:03.173831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=59 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:03.208214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 681/1353] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Jul 18 16:42:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:04.545291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367f355-5a6f-496d-9e59-cb8ab78a90b5 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:04.547668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367f355-5a6f-496d-9e59-cb8ab78a90b5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:04.547985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367f355-5a6f-496d-9e59-cb8ab78a90b5 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:04.549489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 673/1354] 10.222.0.88 () {54 vars in 921 bytes} [Mon Jul 18 16:42:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:04.555024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4240d-2986-4799-84e9-0d1fae33db79 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:04.566647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4240d-2986-4799-84e9-0d1fae33db79 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:04.566647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4240d-2986-4799-84e9-0d1fae33db79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:04.641452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-24c4240d-2986-4799-84e9-0d1fae33db79 None None] MFA Rules not processed for user `22bd4dab1fcd4ad4a8be886646d89242`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:04.642677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-24c4240d-2986-4799-84e9-0d1fae33db79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:04.687289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:04.755614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 682/1355] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Jul 18 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:08.314675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2f9c6-a4f1-4630-a3e1-41e2552ae6ce None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:08.315153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2f9c6-a4f1-4630-a3e1-41e2552ae6ce None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:08.315569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2f9c6-a4f1-4630-a3e1-41e2552ae6ce None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:08.318635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 674/1356] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:08.322834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721798c2-276b-4e8d-abfb-6ce0d61a1bdf None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:08.323292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721798c2-276b-4e8d-abfb-6ce0d61a1bdf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:08.323708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721798c2-276b-4e8d-abfb-6ce0d61a1bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:08.423255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-721798c2-276b-4e8d-abfb-6ce0d61a1bdf None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:08.424408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-721798c2-276b-4e8d-abfb-6ce0d61a1bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:08.538676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 683/1357] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:08.592590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98564556-c1fa-4629-b8fe-446a3854a640 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:08.592908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98564556-c1fa-4629-b8fe-446a3854a640 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:08.593197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98564556-c1fa-4629-b8fe-446a3854a640 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:08.594807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 675/1358] 10.222.0.88 () {54 vars in 962 bytes} [Mon Jul 18 16:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:08.602849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4aa97-f2eb-4b8e-b6ca-b5cc092aea03 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:08.603501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4aa97-f2eb-4b8e-b6ca-b5cc092aea03 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:08.604216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4aa97-f2eb-4b8e-b6ca-b5cc092aea03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:08.623137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:08.691161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8ba4aa97-f2eb-4b8e-b6ca-b5cc092aea03 None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:08.693041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8ba4aa97-f2eb-4b8e-b6ca-b5cc092aea03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:08.820455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 684/1359] 10.222.0.88 () {58 vars in 1059 bytes} [Mon Jul 18 16:42:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:08.829479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98564556-c1fa-4629-b8fe-446a3854a640 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:08.831131 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-98564556-c1fa-4629-b8fe-446a3854a640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:09.002088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98564556-c1fa-4629-b8fe-446a3854a640 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:09.084344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:09.129059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3583e6-88ae-42be-8eca-9d523c78329d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:09.129720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3583e6-88ae-42be-8eca-9d523c78329d service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:09.130001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3583e6-88ae-42be-8eca-9d523c78329d service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:09.130272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3583e6-88ae-42be-8eca-9d523c78329d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:09.131909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5c3583e6-88ae-42be-8eca-9d523c78329d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:09.233080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3583e6-88ae-42be-8eca-9d523c78329d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:09.234783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3583e6-88ae-42be-8eca-9d523c78329d service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:09.256923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 676/1360] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:11.518430 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9047c4-e49b-419c-a6d4-72f475afad30 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:11.518835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9047c4-e49b-419c-a6d4-72f475afad30 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:11.519019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9047c4-e49b-419c-a6d4-72f475afad30 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:11.521186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 685/1361] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:11.525770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6148fa-0d73-4f21-af26-240b12652425 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:11.526220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6148fa-0d73-4f21-af26-240b12652425 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:11.526312 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6148fa-0d73-4f21-af26-240b12652425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:11.604323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3c6148fa-0d73-4f21-af26-240b12652425 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:11.605504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3c6148fa-0d73-4f21-af26-240b12652425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:11.651000 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:11.719276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 677/1362] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:11.768117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9047c4-e49b-419c-a6d4-72f475afad30 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:11.769664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-be9047c4-e49b-419c-a6d4-72f475afad30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:11.870489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:11.971575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9047c4-e49b-419c-a6d4-72f475afad30 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:12.133365 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d86f0b-506b-41d7-a241-5beb5b38f6e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:12.134599 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86f0b-506b-41d7-a241-5beb5b38f6e3 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:12.135090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86f0b-506b-41d7-a241-5beb5b38f6e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:12.135559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86f0b-506b-41d7-a241-5beb5b38f6e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:12.138594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d0d86f0b-506b-41d7-a241-5beb5b38f6e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:12.177752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:12.260451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d86f0b-506b-41d7-a241-5beb5b38f6e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:12.262291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d86f0b-506b-41d7-a241-5beb5b38f6e3 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:12.284122 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 686/1363] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:14.849488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f249988-aefb-4d7e-8ada-983e4ceefb8e None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:14.850031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f249988-aefb-4d7e-8ada-983e4ceefb8e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:14.850031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f249988-aefb-4d7e-8ada-983e4ceefb8e None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:14.853569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 678/1364] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:14.862475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f89f6-43c4-4bdb-a0fe-9dafb0abf645 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:14.862475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f89f6-43c4-4bdb-a0fe-9dafb0abf645 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:14.862475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f89f6-43c4-4bdb-a0fe-9dafb0abf645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:14.941051 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8a1f89f6-43c4-4bdb-a0fe-9dafb0abf645 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:14.942474 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8a1f89f6-43c4-4bdb-a0fe-9dafb0abf645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:15.055298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:15.057485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 687/1365] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:15.103960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f622e-155e-4cd1-8a54-48a8c276769f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:15.104774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f622e-155e-4cd1-8a54-48a8c276769f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:15.105290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f622e-155e-4cd1-8a54-48a8c276769f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:15.106945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 679/1366] 10.222.0.88 () {54 vars in 962 bytes} [Mon Jul 18 16:42:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:15.112984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fe7bd-fca9-4bbd-9f62-9ac00cbb3b2e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:15.113392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fe7bd-fca9-4bbd-9f62-9ac00cbb3b2e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:15.113784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fe7bd-fca9-4bbd-9f62-9ac00cbb3b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:15.203433 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0a5fe7bd-fca9-4bbd-9f62-9ac00cbb3b2e None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:15.204803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0a5fe7bd-fca9-4bbd-9f62-9ac00cbb3b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:15.346838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 688/1367] 10.222.0.88 () {58 vars in 1059 bytes} [Mon Jul 18 16:42:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:15.355837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f622e-155e-4cd1-8a54-48a8c276769f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:15.357521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-af7f622e-155e-4cd1-8a54-48a8c276769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:15.556058 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f622e-155e-4cd1-8a54-48a8c276769f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:15.589813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:15.848988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4d448-bb51-4ad8-851f-bb26e3b27dfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:15.848988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4d448-bb51-4ad8-851f-bb26e3b27dfc service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:15.848988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4d448-bb51-4ad8-851f-bb26e3b27dfc service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:15.848988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4d448-bb51-4ad8-851f-bb26e3b27dfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:15.849501 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-34c4d448-bb51-4ad8-851f-bb26e3b27dfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:15.859832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4d448-bb51-4ad8-851f-bb26e3b27dfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:15.861824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c4d448-bb51-4ad8-851f-bb26e3b27dfc service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:15.881798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 680/1368] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:16.828728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883147e9-aab8-4c8a-8e32-40e15bfd9aca None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:16.828728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883147e9-aab8-4c8a-8e32-40e15bfd9aca None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:16.829587 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883147e9-aab8-4c8a-8e32-40e15bfd9aca None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:16.830611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 689/1369] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:16.835777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba246030-c330-4501-be08-ad8e41b75743 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:16.836176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba246030-c330-4501-be08-ad8e41b75743 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:16.836344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba246030-c330-4501-be08-ad8e41b75743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:16.869212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:16.908465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ba246030-c330-4501-be08-ad8e41b75743 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:16.909709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ba246030-c330-4501-be08-ad8e41b75743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:17.028088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 681/1370] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:17.374794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665228-034e-4edf-8b1f-cae649c62a82 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:17.374794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665228-034e-4edf-8b1f-cae649c62a82 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:17.374794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665228-034e-4edf-8b1f-cae649c62a82 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:17.376775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 690/1371] 10.222.0.88 () {54 vars in 964 bytes} [Mon Jul 18 16:42:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:17.382036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf6625-bf4e-48ce-90fc-457677298bb2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:17.382854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf6625-bf4e-48ce-90fc-457677298bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:17.382854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf6625-bf4e-48ce-90fc-457677298bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:17.482078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e0bf6625-bf4e-48ce-90fc-457677298bb2 None None] MFA Rules not processed for user `809604baaed7486fb1020263e99e9c63`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:17.483315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e0bf6625-bf4e-48ce-90fc-457677298bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:17.537076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:17.575228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 682/1372] 10.222.0.88 () {58 vars in 1061 bytes} [Mon Jul 18 16:42:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:17.576257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2665228-034e-4edf-8b1f-cae649c62a82 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:17.577953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c2665228-034e-4edf-8b1f-cae649c62a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:17.634306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:17.695750 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2665228-034e-4edf-8b1f-cae649c62a82 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:17.780685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d735b3f-8b37-4458-88fc-aa9194c9eebe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:17.781367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735b3f-8b37-4458-88fc-aa9194c9eebe service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:17.781659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735b3f-8b37-4458-88fc-aa9194c9eebe service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:17.781943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d735b3f-8b37-4458-88fc-aa9194c9eebe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:17.783626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9d735b3f-8b37-4458-88fc-aa9194c9eebe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:17.889462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:17.903870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d735b3f-8b37-4458-88fc-aa9194c9eebe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:17.905455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d735b3f-8b37-4458-88fc-aa9194c9eebe service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:17.928685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 691/1373] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:42:18.257965 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63ef82-3496-4256-acd2-facbad63cc5c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:18.257965 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63ef82-3496-4256-acd2-facbad63cc5c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:18.257965 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63ef82-3496-4256-acd2-facbad63cc5c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:18.267437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ef613-8d09-45e4-815b-78336daf5263 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:18.267437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ef613-8d09-45e4-815b-78336daf5263 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:18.267437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ef613-8d09-45e4-815b-78336daf5263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:18.271821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 683/1374] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:18 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:18.362951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ba1ef613-8d09-45e4-815b-78336daf5263 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:18.364264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ba1ef613-8d09-45e4-815b-78336daf5263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:18.481752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 692/1375] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:18.527173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63ef82-3496-4256-acd2-facbad63cc5c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:18.528794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1a63ef82-3496-4256-acd2-facbad63cc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:18.690740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63ef82-3496-4256-acd2-facbad63cc5c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:18.746446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:18.810939 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa2fa3-9617-48f0-93db-b0d32f5b006f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:18.811749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa2fa3-9617-48f0-93db-b0d32f5b006f service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:18.811875 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa2fa3-9617-48f0-93db-b0d32f5b006f service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:18.812141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa2fa3-9617-48f0-93db-b0d32f5b006f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:18.813845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-98fa2fa3-9617-48f0-93db-b0d32f5b006f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:18.907899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa2fa3-9617-48f0-93db-b0d32f5b006f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:18.909549 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa2fa3-9617-48f0-93db-b0d32f5b006f service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:18.931432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 684/1376] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:21.141027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24888cd-fa25-4016-82d5-fa0f9093dcd6 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:21.141883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24888cd-fa25-4016-82d5-fa0f9093dcd6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:21.142317 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24888cd-fa25-4016-82d5-fa0f9093dcd6 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:21.144144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 693/1377] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:21.150482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521034b4-7b24-4110-8e45-8a6a8484b1fe None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:21.150947 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521034b4-7b24-4110-8e45-8a6a8484b1fe None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:21.151062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521034b4-7b24-4110-8e45-8a6a8484b1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:21.216981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-521034b4-7b24-4110-8e45-8a6a8484b1fe None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:21.218163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-521034b4-7b24-4110-8e45-8a6a8484b1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:21.225107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:21.316874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 685/1378] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:21.361788 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24888cd-fa25-4016-82d5-fa0f9093dcd6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:21.370784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f24888cd-fa25-4016-82d5-fa0f9093dcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:21.433586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:21.561235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24888cd-fa25-4016-82d5-fa0f9093dcd6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:21.713255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fa464-5e56-4f52-b692-e1115c4eb9b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:21.714045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fa464-5e56-4f52-b692-e1115c4eb9b9 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:21.714436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fa464-5e56-4f52-b692-e1115c4eb9b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:21.714842 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fa464-5e56-4f52-b692-e1115c4eb9b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:21.716803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d07fa464-5e56-4f52-b692-e1115c4eb9b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:21.738336 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:21.817142 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fa464-5e56-4f52-b692-e1115c4eb9b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:21.818766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fa464-5e56-4f52-b692-e1115c4eb9b9 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:21.840235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 694/1379] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:24.045632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfea10-ce6e-49b3-a427-3a00e12a342f None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:24.046130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfea10-ce6e-49b3-a427-3a00e12a342f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:24.046241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfea10-ce6e-49b3-a427-3a00e12a342f None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:24.047588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 686/1380] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:24.052744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f183b-d606-4fe8-b5fe-7731128c76f5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:24.053188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f183b-d606-4fe8-b5fe-7731128c76f5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:24.053329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f183b-d606-4fe8-b5fe-7731128c76f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:24.118706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1f1f183b-d606-4fe8-b5fe-7731128c76f5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:24.119932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1f1f183b-d606-4fe8-b5fe-7731128c76f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:24.211097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:24.217860 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 695/1381] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:24.264541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfea10-ce6e-49b3-a427-3a00e12a342f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:24.266241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f6dfea10-ce6e-49b3-a427-3a00e12a342f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:24.425495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:24.437059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfea10-ce6e-49b3-a427-3a00e12a342f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:24.571763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bffb9-d5b8-4403-8a33-a897b59b058e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:24.572482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bffb9-d5b8-4403-8a33-a897b59b058e service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:24.572766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bffb9-d5b8-4403-8a33-a897b59b058e service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:24.573081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bffb9-d5b8-4403-8a33-a897b59b058e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:24.574750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3c3bffb9-d5b8-4403-8a33-a897b59b058e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:24.680933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bffb9-d5b8-4403-8a33-a897b59b058e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:24.682870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bffb9-d5b8-4403-8a33-a897b59b058e service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:24.695134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:24.711771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 687/1382] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:26.870758 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaed13-4e8c-48ee-83be-6763071db97e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:26.871316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaed13-4e8c-48ee-83be-6763071db97e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:26.871526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaed13-4e8c-48ee-83be-6763071db97e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:26.873239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 696/1383] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:26.878694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec8a3-8442-4bb0-b9f1-4a2d704e994f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:26.879134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec8a3-8442-4bb0-b9f1-4a2d704e994f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:26.879266 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ec8a3-8442-4bb0-b9f1-4a2d704e994f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:26.967720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-db6ec8a3-8442-4bb0-b9f1-4a2d704e994f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:26.969349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-db6ec8a3-8442-4bb0-b9f1-4a2d704e994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:27.082841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 688/1384] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:27.127273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecaed13-4e8c-48ee-83be-6763071db97e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:27.129190 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-eecaed13-4e8c-48ee-83be-6763071db97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:27.316330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecaed13-4e8c-48ee-83be-6763071db97e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:27.389607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:27.445410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a261a0-ea33-4449-9032-efd59728ac6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:27.446583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a261a0-ea33-4449-9032-efd59728ac6c service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:27.446867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a261a0-ea33-4449-9032-efd59728ac6c service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:27.447172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a261a0-ea33-4449-9032-efd59728ac6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:27.448850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-55a261a0-ea33-4449-9032-efd59728ac6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:27.549076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a261a0-ea33-4449-9032-efd59728ac6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:27.550701 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a261a0-ea33-4449-9032-efd59728ac6c service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:27.569645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 697/1385] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:29.683908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eeefe-80f6-418a-b299-4d29ba08b1e3 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:29.684458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eeefe-80f6-418a-b299-4d29ba08b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:29.684458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eeefe-80f6-418a-b299-4d29ba08b1e3 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:29.685905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 689/1386] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:29.691528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0707a-6e31-45c2-8534-d3c8e988fbc1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:29.692149 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0707a-6e31-45c2-8534-d3c8e988fbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:29.692149 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0707a-6e31-45c2-8534-d3c8e988fbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:29.771608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f0f0707a-6e31-45c2-8534-d3c8e988fbc1 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:29.773442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f0f0707a-6e31-45c2-8534-d3c8e988fbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:29.817874 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:29.891495 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 698/1387] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:29.940791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932eeefe-80f6-418a-b299-4d29ba08b1e3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:29.942433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-932eeefe-80f6-418a-b299-4d29ba08b1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:30.023072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:30.097144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932eeefe-80f6-418a-b299-4d29ba08b1e3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:30.220040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500c071-2470-4784-90b5-07f02476f13a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:30.220993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500c071-2470-4784-90b5-07f02476f13a service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:30.221508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500c071-2470-4784-90b5-07f02476f13a service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:30.222007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500c071-2470-4784-90b5-07f02476f13a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:30.224664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b500c071-2470-4784-90b5-07f02476f13a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:30.275391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:30.334999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500c071-2470-4784-90b5-07f02476f13a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:30.337263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500c071-2470-4784-90b5-07f02476f13a service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:30.356219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 690/1388] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:32.701838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde44e-9021-4fcb-8f6b-6a20ad97eac3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:32.702399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde44e-9021-4fcb-8f6b-6a20ad97eac3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:32.702399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cde44e-9021-4fcb-8f6b-6a20ad97eac3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:32.703846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 699/1389] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:32.709117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14212e-aaa4-468d-a91d-6d90343914ad None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:32.709539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14212e-aaa4-468d-a91d-6d90343914ad None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:32.709746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14212e-aaa4-468d-a91d-6d90343914ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:32.800769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3d14212e-aaa4-468d-a91d-6d90343914ad None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:32.802908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3d14212e-aaa4-468d-a91d-6d90343914ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:32.946481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 691/1390] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:32.999572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cde44e-9021-4fcb-8f6b-6a20ad97eac3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:33.001173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-67cde44e-9021-4fcb-8f6b-6a20ad97eac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:33.175790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cde44e-9021-4fcb-8f6b-6a20ad97eac3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:33.195748 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:33.335639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a35785-f915-4e65-8af5-58ad8145cb76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:33.337382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a35785-f915-4e65-8af5-58ad8145cb76 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:33.337753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a35785-f915-4e65-8af5-58ad8145cb76 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:33.338032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a35785-f915-4e65-8af5-58ad8145cb76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:33.340056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-41a35785-f915-4e65-8af5-58ad8145cb76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:33.454374 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a35785-f915-4e65-8af5-58ad8145cb76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:33.455997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a35785-f915-4e65-8af5-58ad8145cb76 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:33.477790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 700/1391] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:33.495184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87054b3-ed12-4b40-a201-beb88c0eccdd None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:33.495476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87054b3-ed12-4b40-a201-beb88c0eccdd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:33.495760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87054b3-ed12-4b40-a201-beb88c0eccdd None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:33.497211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 692/1392] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:33.503144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8e0a0-9c8a-484f-b04d-71514cba8eb3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:33.503429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8e0a0-9c8a-484f-b04d-71514cba8eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:33.503706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8e0a0-9c8a-484f-b04d-71514cba8eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:33.530843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:33.575567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0fd8e0a0-9c8a-484f-b04d-71514cba8eb3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:33.576781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0fd8e0a0-9c8a-484f-b04d-71514cba8eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:33.698813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 701/1393] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:33.762278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e590e6c-3b1d-496c-89dc-2d72dccf3339 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:33.762278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e590e6c-3b1d-496c-89dc-2d72dccf3339 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:33.762278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e590e6c-3b1d-496c-89dc-2d72dccf3339 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:33.762278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 693/1394] 10.222.0.88 () {54 vars in 961 bytes} [Mon Jul 18 16:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:33.770349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeae57-31b1-443c-879f-cb09db0a10ad None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:33.770349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeae57-31b1-443c-879f-cb09db0a10ad None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:33.770349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeae57-31b1-443c-879f-cb09db0a10ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:33.862245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e6eeae57-31b1-443c-879f-cb09db0a10ad None None] MFA Rules not processed for user `54c7869983bf4534b3e730af0185eac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:33.863410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e6eeae57-31b1-443c-879f-cb09db0a10ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:33.914220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:33.964046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 702/1395] 10.222.0.88 () {58 vars in 1058 bytes} [Mon Jul 18 16:42:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:33.972330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e590e6c-3b1d-496c-89dc-2d72dccf3339 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:33.974152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7e590e6c-3b1d-496c-89dc-2d72dccf3339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:33.989647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:34.093238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e590e6c-3b1d-496c-89dc-2d72dccf3339 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:34.170019 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7119dfa-790e-42dd-a03a-2923568e5824 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54c7869983bf4534b3e730af0185eac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:34.170587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7119dfa-790e-42dd-a03a-2923568e5824 service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:34.170873 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7119dfa-790e-42dd-a03a-2923568e5824 service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:34.171151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7119dfa-790e-42dd-a03a-2923568e5824 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:34.172838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d7119dfa-790e-42dd-a03a-2923568e5824 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:34.249869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:34.313455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7119dfa-790e-42dd-a03a-2923568e5824 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:34.314930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7119dfa-790e-42dd-a03a-2923568e5824 service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:34.339857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 694/1396] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:35.257655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc12852-736b-4ec2-8fe5-c98f1c757e60 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:35.258282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc12852-736b-4ec2-8fe5-c98f1c757e60 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:35.258282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc12852-736b-4ec2-8fe5-c98f1c757e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:35.327632 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-dfc12852-736b-4ec2-8fe5-c98f1c757e60 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:35.328841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dfc12852-736b-4ec2-8fe5-c98f1c757e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:35.422119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 703/1397] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:42:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:35.466427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae9a6c-d525-45f0-bf2c-fbf026230057 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:35.470051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae9a6c-d525-45f0-bf2c-fbf026230057 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:35.470051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae9a6c-d525-45f0-bf2c-fbf026230057 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:35.470051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 695/1398] 10.222.0.88 () {54 vars in 943 bytes} [Mon Jul 18 16:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:42:35.475580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8063fa-2e91-43c2-bb29-c3850458a2c9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:35.475913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8063fa-2e91-43c2-bb29-c3850458a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:35.476197 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8063fa-2e91-43c2-bb29-c3850458a2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:35.547419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:35.555364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9b8063fa-2e91-43c2-bb29-c3850458a2c9 None None] MFA Rules not processed for user `bf341f1bb8944a3e9331678100b941a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:35.556598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9b8063fa-2e91-43c2-bb29-c3850458a2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:35.666100 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 704/1399] 10.222.0.88 () {58 vars in 1040 bytes} [Mon Jul 18 16:42:35 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:35.675508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae9a6c-d525-45f0-bf2c-fbf026230057 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:35.677108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-06ae9a6c-d525-45f0-bf2c-fbf026230057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:35.821408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae9a6c-d525-45f0-bf2c-fbf026230057 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:35.860951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:35.899406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac2e6f-6577-44c2-ac2f-625ef0c0019c None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:35.899692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac2e6f-6577-44c2-ac2f-625ef0c0019c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:35.899972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac2e6f-6577-44c2-ac2f-625ef0c0019c None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:35.901529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 705/1400] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:42:35.906551 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6448200-61f9-4881-904a-0b99608b3fdf None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:35.906833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6448200-61f9-4881-904a-0b99608b3fdf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:35.907530 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6448200-61f9-4881-904a-0b99608b3fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:35.938016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29340c04-3402-4f7c-adba-ec9261377d9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:35.938016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29340c04-3402-4f7c-adba-ec9261377d9a service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:35.938016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29340c04-3402-4f7c-adba-ec9261377d9a service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:35.938016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29340c04-3402-4f7c-adba-ec9261377d9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:35.938467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-29340c04-3402-4f7c-adba-ec9261377d9a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:35.983041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f6448200-61f9-4881-904a-0b99608b3fdf None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:35.984282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f6448200-61f9-4881-904a-0b99608b3fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:36.026098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29340c04-3402-4f7c-adba-ec9261377d9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:36.027720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29340c04-3402-4f7c-adba-ec9261377d9a service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:36.035899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 696/1401] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:42:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:42:36.049595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:36.087974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 706/1402] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:42:36.134044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29340c04-3402-4f7c-adba-ec9261377d9a service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:36.135261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-29340c04-3402-4f7c-adba-ec9261377d9a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:36.201485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:36.228291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29340c04-3402-4f7c-adba-ec9261377d9a service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:36.358541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db1ba5-e901-4a87-b658-54fece2e0d0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:36.359120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db1ba5-e901-4a87-b658-54fece2e0d0f service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:36.359407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db1ba5-e901-4a87-b658-54fece2e0d0f service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:36.359710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db1ba5-e901-4a87-b658-54fece2e0d0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:36.361437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-42db1ba5-e901-4a87-b658-54fece2e0d0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:36.454862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:36.458482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db1ba5-e901-4a87-b658-54fece2e0d0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:36.460076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db1ba5-e901-4a87-b658-54fece2e0d0f service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:36.482494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 697/1403] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:39.699519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa231a-0183-4750-8823-ed6abcafcf7f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:39.700058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa231a-0183-4750-8823-ed6abcafcf7f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:39.700058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa231a-0183-4750-8823-ed6abcafcf7f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:39.704918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 707/1404] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 16:42:39.706854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a9e7e-1f32-4e88-928b-c7559c8f1050 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:39.707318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a9e7e-1f32-4e88-928b-c7559c8f1050 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:39.707411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a9e7e-1f32-4e88-928b-c7559c8f1050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:39.780572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-936a9e7e-1f32-4e88-928b-c7559c8f1050 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:39.782238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-936a9e7e-1f32-4e88-928b-c7559c8f1050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:39.891857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 698/1405] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:39.938887 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa231a-0183-4750-8823-ed6abcafcf7f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:39.940549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-34aa231a-0183-4750-8823-ed6abcafcf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:40.111204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa231a-0183-4750-8823-ed6abcafcf7f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:40.163229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:40.243692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c78b85-4254-4683-baa8-952af17b5b48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:40.244368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78b85-4254-4683-baa8-952af17b5b48 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:40.244654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78b85-4254-4683-baa8-952af17b5b48 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:40.244949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c78b85-4254-4683-baa8-952af17b5b48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:40.246694 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-02c78b85-4254-4683-baa8-952af17b5b48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:40.360234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c78b85-4254-4683-baa8-952af17b5b48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:40.362289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c78b85-4254-4683-baa8-952af17b5b48 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:40.384458 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 708/1406] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:42.670690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536d09-db58-4f8e-931b-8d7d89c973c9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:42.671282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536d09-db58-4f8e-931b-8d7d89c973c9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:42.671282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b536d09-db58-4f8e-931b-8d7d89c973c9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:42.672934 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 699/1407] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:42:42.678388 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922e0ad-32c2-40cb-9827-ba5148823819 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:42.678849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922e0ad-32c2-40cb-9827-ba5148823819 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:42.678959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922e0ad-32c2-40cb-9827-ba5148823819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:42.759547 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5922e0ad-32c2-40cb-9827-ba5148823819 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:42:42.761437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5922e0ad-32c2-40cb-9827-ba5148823819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:42.767605 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:42.888205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 709/1408] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:42:42.940095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b536d09-db58-4f8e-931b-8d7d89c973c9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:42:42.941711 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0b536d09-db58-4f8e-931b-8d7d89c973c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:43.023983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:43.116852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b536d09-db58-4f8e-931b-8d7d89c973c9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:42:43.252498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db76596-7364-4dec-8bcb-561c9bab9b6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:42:43.252498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db76596-7364-4dec-8bcb-561c9bab9b6d service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:42:43.252498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db76596-7364-4dec-8bcb-561c9bab9b6d service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:42:43.252498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db76596-7364-4dec-8bcb-561c9bab9b6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:42:43.253420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6db76596-7364-4dec-8bcb-561c9bab9b6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:42:43.283899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:43.354182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db76596-7364-4dec-8bcb-561c9bab9b6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:42:43.355881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db76596-7364-4dec-8bcb-561c9bab9b6d service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:42:43.376774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 700/1409] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:42:52.886370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=41 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:42:53.374008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=34 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:00.576395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a60a9d-784d-41fa-adca-9fb16218c5de None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:00.577010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a60a9d-784d-41fa-adca-9fb16218c5de None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:00.577010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a60a9d-784d-41fa-adca-9fb16218c5de None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:00.578422 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 710/1410] 10.222.0.88 () {54 vars in 918 bytes} [Mon Jul 18 16:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:00.583849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01008280-a434-4b43-bd99-b8d1d167a17d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:00.584300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01008280-a434-4b43-bd99-b8d1d167a17d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:00.584406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01008280-a434-4b43-bd99-b8d1d167a17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:00.778853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-01008280-a434-4b43-bd99-b8d1d167a17d None None] MFA Rules not processed for user `22bd4dab1fcd4ad4a8be886646d89242`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:00.780108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-01008280-a434-4b43-bd99-b8d1d167a17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:00.892305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 701/1411] 10.222.0.88 () {58 vars in 1015 bytes} [Mon Jul 18 16:43:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:00.906717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715a239-ab4c-4262-82ba-41439ed794ee None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:00.907005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715a239-ab4c-4262-82ba-41439ed794ee None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:00.907292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715a239-ab4c-4262-82ba-41439ed794ee None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:00.908649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 711/1412] 10.222.0.88 () {54 vars in 918 bytes} [Mon Jul 18 16:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:00.913448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f93cae-1bd2-49de-90d8-0cd90363cf35 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:00.913738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f93cae-1bd2-49de-90d8-0cd90363cf35 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:00.914013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f93cae-1bd2-49de-90d8-0cd90363cf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:00.990740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:00.993851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-76f93cae-1bd2-49de-90d8-0cd90363cf35 None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:00.995121 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-76f93cae-1bd2-49de-90d8-0cd90363cf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:01.110545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 702/1413] 10.222.0.88 () {58 vars in 1015 bytes} [Mon Jul 18 16:43:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:03.675901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20a1aa-6fdf-48e6-96a6-a080f631f060 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:03.676473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20a1aa-6fdf-48e6-96a6-a080f631f060 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:03.676473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20a1aa-6fdf-48e6-96a6-a080f631f060 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:03.677883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:03.683661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62722d-808d-42dd-bab5-bcc4f759ef64 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:03.684113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62722d-808d-42dd-bab5-bcc4f759ef64 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:03.684236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62722d-808d-42dd-bab5-bcc4f759ef64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:03.750432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4a62722d-808d-42dd-bab5-bcc4f759ef64 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:03.751683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4a62722d-808d-42dd-bab5-bcc4f759ef64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:03.807196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:03.849128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:03.892839 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20a1aa-6fdf-48e6-96a6-a080f631f060 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:03.894523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0b20a1aa-6fdf-48e6-96a6-a080f631f060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:04.041569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20a1aa-6fdf-48e6-96a6-a080f631f060 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:04.115057 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:04.155403 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c99f8-9900-417f-b5a8-96248c792ec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:04.156053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c99f8-9900-417f-b5a8-96248c792ec1 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:04.156340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c99f8-9900-417f-b5a8-96248c792ec1 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:04.156616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c99f8-9900-417f-b5a8-96248c792ec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:04.158340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c13c99f8-9900-417f-b5a8-96248c792ec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:04.250643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c99f8-9900-417f-b5a8-96248c792ec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:04.252339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c99f8-9900-417f-b5a8-96248c792ec1 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:04.270608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 713/1416] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:05.208246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2f363-0531-4484-bd02-2f794ee13d6b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:05.208772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2f363-0531-4484-bd02-2f794ee13d6b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:05.208772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2f363-0531-4484-bd02-2f794ee13d6b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:05.210113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:05.215132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc2e6c-f9fa-49df-8fa0-5c84c9b17298 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:05.215573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc2e6c-f9fa-49df-8fa0-5c84c9b17298 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:05.215686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc2e6c-f9fa-49df-8fa0-5c84c9b17298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:05.291730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4cfc2e6c-f9fa-49df-8fa0-5c84c9b17298 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:05.292963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4cfc2e6c-f9fa-49df-8fa0-5c84c9b17298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:05.319858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:05.414438 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:43:05.572637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2f363-0531-4484-bd02-2f794ee13d6b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:05.574259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-65a2f363-0531-4484-bd02-2f794ee13d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:05.723142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2f363-0531-4484-bd02-2f794ee13d6b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:05.767267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:05.846982 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d999d5-a8fe-4735-99ab-1cfaba8d9086 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:05.847800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d999d5-a8fe-4735-99ab-1cfaba8d9086 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:05.848078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d999d5-a8fe-4735-99ab-1cfaba8d9086 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:05.848349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d999d5-a8fe-4735-99ab-1cfaba8d9086 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:05.850040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c3d999d5-a8fe-4735-99ab-1cfaba8d9086 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:05.938291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d999d5-a8fe-4735-99ab-1cfaba8d9086 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:05.939903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d999d5-a8fe-4735-99ab-1cfaba8d9086 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:05.958550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 705/1419] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:06.853695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541b175-634c-4ce4-b0a1-2953f69fece3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:06.854291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541b175-634c-4ce4-b0a1-2953f69fece3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:06.854617 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541b175-634c-4ce4-b0a1-2953f69fece3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:06.856016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:06.861254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd2184-e4b7-4792-b043-261c703f728e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:06.861697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd2184-e4b7-4792-b043-261c703f728e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:06.861850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd2184-e4b7-4792-b043-261c703f728e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:06.916203 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:06.939125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0ebd2184-e4b7-4792-b043-261c703f728e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:06.940499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0ebd2184-e4b7-4792-b043-261c703f728e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:07.048547 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:07.203801 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541b175-634c-4ce4-b0a1-2953f69fece3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:07.205388 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d541b175-634c-4ce4-b0a1-2953f69fece3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:07.358885 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541b175-634c-4ce4-b0a1-2953f69fece3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:07.372206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:07.479296 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebf728d-160c-400f-acc8-16f25e8d6510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:07.479861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf728d-160c-400f-acc8-16f25e8d6510 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:07.480147 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf728d-160c-400f-acc8-16f25e8d6510 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:07.480427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf728d-160c-400f-acc8-16f25e8d6510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:07.482185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5ebf728d-160c-400f-acc8-16f25e8d6510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:07.579375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf728d-160c-400f-acc8-16f25e8d6510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:07.581084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebf728d-160c-400f-acc8-16f25e8d6510 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:07.601904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 716/1422] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:08.480295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e1ed-6a19-4ab3-815c-59cbdcc8c88d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:08.480854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e1ed-6a19-4ab3-815c-59cbdcc8c88d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:08.480854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e1ed-6a19-4ab3-815c-59cbdcc8c88d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:08.482320 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:08.487431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c656d0-e312-4dd7-9a12-7ed5ae131cc7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:08.487867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c656d0-e312-4dd7-9a12-7ed5ae131cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:08.487981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c656d0-e312-4dd7-9a12-7ed5ae131cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:08.517393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:08.576667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-17c656d0-e312-4dd7-9a12-7ed5ae131cc7 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:08.577949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-17c656d0-e312-4dd7-9a12-7ed5ae131cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:08.675254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:08.825192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145e1ed-6a19-4ab3-815c-59cbdcc8c88d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:08.829916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d145e1ed-6a19-4ab3-815c-59cbdcc8c88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:08.943723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:08.981681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145e1ed-6a19-4ab3-815c-59cbdcc8c88d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:09.094914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7bfa83-9e2d-4126-a2d3-3d9ad5b5201a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:09.095561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bfa83-9e2d-4126-a2d3-3d9ad5b5201a service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:09.095845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bfa83-9e2d-4126-a2d3-3d9ad5b5201a service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:09.096123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bfa83-9e2d-4126-a2d3-3d9ad5b5201a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:09.098377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3b7bfa83-9e2d-4126-a2d3-3d9ad5b5201a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:09.174518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:09.187977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7bfa83-9e2d-4126-a2d3-3d9ad5b5201a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:09.189552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7bfa83-9e2d-4126-a2d3-3d9ad5b5201a service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:09.207758 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 708/1425] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:10.072250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f2568-0ac1-4db1-926d-ee67c3aa04f2 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:10.072805 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f2568-0ac1-4db1-926d-ee67c3aa04f2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:10.072805 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f2568-0ac1-4db1-926d-ee67c3aa04f2 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:10.074269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:10.079375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108a6a8-0577-4a7e-900f-9af7d3e9aa73 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:10.079794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108a6a8-0577-4a7e-900f-9af7d3e9aa73 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:10.080092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108a6a8-0577-4a7e-900f-9af7d3e9aa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:10.154839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6108a6a8-0577-4a7e-900f-9af7d3e9aa73 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:10.156067 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6108a6a8-0577-4a7e-900f-9af7d3e9aa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:10.255855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:10.407342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f2568-0ac1-4db1-926d-ee67c3aa04f2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:10.408923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-599f2568-0ac1-4db1-926d-ee67c3aa04f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:10.495519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:10.558878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f2568-0ac1-4db1-926d-ee67c3aa04f2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:10.676561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf37be0-5ff2-422c-ac43-f24aee6ecc49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:10.677262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf37be0-5ff2-422c-ac43-f24aee6ecc49 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:10.677534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf37be0-5ff2-422c-ac43-f24aee6ecc49 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:10.677812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf37be0-5ff2-422c-ac43-f24aee6ecc49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:10.679495 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ddf37be0-5ff2-422c-ac43-f24aee6ecc49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:10.730319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:10.773369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf37be0-5ff2-422c-ac43-f24aee6ecc49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:10.775517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf37be0-5ff2-422c-ac43-f24aee6ecc49 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:10.794074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 719/1428] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:11.670132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef33a4f-44f9-4291-986d-a727baa972e1 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:11.670682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef33a4f-44f9-4291-986d-a727baa972e1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:11.670682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef33a4f-44f9-4291-986d-a727baa972e1 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:11.672106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:11.677397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe01e9-7269-4ad5-95e5-464ede3d315d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:11.677847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe01e9-7269-4ad5-95e5-464ede3d315d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:11.677947 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe01e9-7269-4ad5-95e5-464ede3d315d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:11.752277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0afe01e9-7269-4ad5-95e5-464ede3d315d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:11.753537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0afe01e9-7269-4ad5-95e5-464ede3d315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:11.850905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:12.000601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef33a4f-44f9-4291-986d-a727baa972e1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:12.002158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2ef33a4f-44f9-4291-986d-a727baa972e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:12.050705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:12.161620 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef33a4f-44f9-4291-986d-a727baa972e1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:12.285177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c197c6c-d30d-4a7c-83ef-ed3d799cb896 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:12.285835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c197c6c-d30d-4a7c-83ef-ed3d799cb896 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:12.286120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c197c6c-d30d-4a7c-83ef-ed3d799cb896 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:12.286398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c197c6c-d30d-4a7c-83ef-ed3d799cb896 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:12.288085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8c197c6c-d30d-4a7c-83ef-ed3d799cb896 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:12.295793 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:12.385488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c197c6c-d30d-4a7c-83ef-ed3d799cb896 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:12.387112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c197c6c-d30d-4a7c-83ef-ed3d799cb896 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:12.405680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 711/1431] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:13.277992 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d479c77-8f8b-4838-8d6a-42efa3ab9d7f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:13.278530 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d479c77-8f8b-4838-8d6a-42efa3ab9d7f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:13.278628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d479c77-8f8b-4838-8d6a-42efa3ab9d7f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:13.280170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:13.285089 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f67e64-048f-4f7a-9a30-48d39ab2a807 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:13.285546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f67e64-048f-4f7a-9a30-48d39ab2a807 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:13.285654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f67e64-048f-4f7a-9a30-48d39ab2a807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:13.360649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-41f67e64-048f-4f7a-9a30-48d39ab2a807 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:13.361969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-41f67e64-048f-4f7a-9a30-48d39ab2a807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:13.431823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:13.460415 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:13.608838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d479c77-8f8b-4838-8d6a-42efa3ab9d7f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:13.610377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1d479c77-8f8b-4838-8d6a-42efa3ab9d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:13.629496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:13.760540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d479c77-8f8b-4838-8d6a-42efa3ab9d7f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:13.847509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:13.874957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44e5cf-b771-46df-b0d9-aff3c2b19de8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:13.875525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44e5cf-b771-46df-b0d9-aff3c2b19de8 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:13.875806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44e5cf-b771-46df-b0d9-aff3c2b19de8 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:13.876081 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44e5cf-b771-46df-b0d9-aff3c2b19de8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:13.877782 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ba44e5cf-b771-46df-b0d9-aff3c2b19de8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:13.967893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44e5cf-b771-46df-b0d9-aff3c2b19de8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:13.969557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44e5cf-b771-46df-b0d9-aff3c2b19de8 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:13.988317 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 722/1434] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:14.952221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc63bce-d654-4d8a-b45a-5d193c8959b4 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:14.952753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc63bce-d654-4d8a-b45a-5d193c8959b4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:14.952849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc63bce-d654-4d8a-b45a-5d193c8959b4 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:14.954206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:14.960725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849c48d-d521-48f3-9eb5-8b7926282f57 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:14.961822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849c48d-d521-48f3-9eb5-8b7926282f57 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:14.961913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849c48d-d521-48f3-9eb5-8b7926282f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:15.047623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e849c48d-d521-48f3-9eb5-8b7926282f57 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:15.048866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e849c48d-d521-48f3-9eb5-8b7926282f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:15.092540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:15.153398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:15.308786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc63bce-d654-4d8a-b45a-5d193c8959b4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:15.310339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9dc63bce-d654-4d8a-b45a-5d193c8959b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:15.459726 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc63bce-d654-4d8a-b45a-5d193c8959b4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:15.517354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:15.578095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca12ba4-1880-47bc-b4ee-b7622128331f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:15.578796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca12ba4-1880-47bc-b4ee-b7622128331f service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:15.579170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca12ba4-1880-47bc-b4ee-b7622128331f service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:15.579561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca12ba4-1880-47bc-b4ee-b7622128331f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:15.581408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1ca12ba4-1880-47bc-b4ee-b7622128331f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:15.674455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca12ba4-1880-47bc-b4ee-b7622128331f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:15.676064 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca12ba4-1880-47bc-b4ee-b7622128331f service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:15.694435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 714/1437] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:16.607119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befff20-da40-4624-bf94-bc8185e6b7e0 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:16.607649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befff20-da40-4624-bf94-bc8185e6b7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:16.607649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befff20-da40-4624-bf94-bc8185e6b7e0 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:16.608973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:16.614025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c5a56-ddb3-4ec4-a8f6-7e63cd3ab662 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:16.614420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c5a56-ddb3-4ec4-a8f6-7e63cd3ab662 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:16.614590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c5a56-ddb3-4ec4-a8f6-7e63cd3ab662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:16.689248 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fd8c5a56-ddb3-4ec4-a8f6-7e63cd3ab662 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:16.690467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fd8c5a56-ddb3-4ec4-a8f6-7e63cd3ab662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:16.692508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:16.788577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:43:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:16.941234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befff20-da40-4624-bf94-bc8185e6b7e0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:16.942765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9befff20-da40-4624-bf94-bc8185e6b7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:17.090820 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befff20-da40-4624-bf94-bc8185e6b7e0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:17.120366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:17.227078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a57931-4206-45f6-bb5d-42d8b6731ce5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:17.227695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a57931-4206-45f6-bb5d-42d8b6731ce5 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:17.227974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a57931-4206-45f6-bb5d-42d8b6731ce5 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:17.228244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a57931-4206-45f6-bb5d-42d8b6731ce5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:17.229988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f5a57931-4206-45f6-bb5d-42d8b6731ce5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:17.330771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a57931-4206-45f6-bb5d-42d8b6731ce5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:17.332399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a57931-4206-45f6-bb5d-42d8b6731ce5 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:17.352146 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 725/1440] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:19.345832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2872773-6b39-4b91-8bd5-41eac233778a None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:19.346459 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2872773-6b39-4b91-8bd5-41eac233778a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:19.346459 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2872773-6b39-4b91-8bd5-41eac233778a None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:19.348095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:19.353327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a1d13-91d1-4795-9652-9a8a9d14aadc None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:19.353749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a1d13-91d1-4795-9652-9a8a9d14aadc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:19.353867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a1d13-91d1-4795-9652-9a8a9d14aadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:19.394540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:19.420948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-927a1d13-91d1-4795-9652-9a8a9d14aadc None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:19.422212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-927a1d13-91d1-4795-9652-9a8a9d14aadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:19.522832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:19.561407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2872773-6b39-4b91-8bd5-41eac233778a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:19.562974 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d2872773-6b39-4b91-8bd5-41eac233778a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:19.684001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2872773-6b39-4b91-8bd5-41eac233778a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:19.717561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:19.764389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7deeb-9bd6-497c-ab46-55e1fae7b402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:19.765166 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7deeb-9bd6-497c-ab46-55e1fae7b402 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:19.766031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7deeb-9bd6-497c-ab46-55e1fae7b402 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:19.766031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7deeb-9bd6-497c-ab46-55e1fae7b402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:19.767717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f7d7deeb-9bd6-497c-ab46-55e1fae7b402 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:19.884520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7deeb-9bd6-497c-ab46-55e1fae7b402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:19.886125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7deeb-9bd6-497c-ab46-55e1fae7b402 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:19.906635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 717/1443] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:22.344226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c8842-1435-4a46-856a-d9f349af24d4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:22.344830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c8842-1435-4a46-856a-d9f349af24d4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:22.344990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c8842-1435-4a46-856a-d9f349af24d4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:22.346386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:22.353317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9f573-256e-4ef3-a1e9-33be4f804b98 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:22.353828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9f573-256e-4ef3-a1e9-33be4f804b98 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:22.353924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9f573-256e-4ef3-a1e9-33be4f804b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:22.422592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2de9f573-256e-4ef3-a1e9-33be4f804b98 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:22.433938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2de9f573-256e-4ef3-a1e9-33be4f804b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:22.437722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:22.529812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:22.569016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c8842-1435-4a46-856a-d9f349af24d4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:22.570689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-162c8842-1435-4a46-856a-d9f349af24d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:22.679847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c8842-1435-4a46-856a-d9f349af24d4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:22.692721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:22.772553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bd84d-e46d-4e4c-99fa-f613832763ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:22.777184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bd84d-e46d-4e4c-99fa-f613832763ae service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:22.777460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bd84d-e46d-4e4c-99fa-f613832763ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:22.777802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bd84d-e46d-4e4c-99fa-f613832763ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:22.780416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-602bd84d-e46d-4e4c-99fa-f613832763ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:22.907685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bd84d-e46d-4e4c-99fa-f613832763ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:22.909134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bd84d-e46d-4e4c-99fa-f613832763ae service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:22.934422 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 728/1446] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:25.502907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef842b-892b-43a7-9bbe-5ecf77b63c4b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:25.503410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef842b-892b-43a7-9bbe-5ecf77b63c4b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:25.503522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef842b-892b-43a7-9bbe-5ecf77b63c4b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:25.504861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:25.510878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72565f-841d-474f-83b8-0ed3f8d9f1ce None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:25.511323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72565f-841d-474f-83b8-0ed3f8d9f1ce None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:25.511432 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72565f-841d-474f-83b8-0ed3f8d9f1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:25.560331 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:25.579454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8c72565f-841d-474f-83b8-0ed3f8d9f1ce None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:25.580827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8c72565f-841d-474f-83b8-0ed3f8d9f1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:25.677643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:25.716815 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef842b-892b-43a7-9bbe-5ecf77b63c4b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:25.718395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ebef842b-892b-43a7-9bbe-5ecf77b63c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:25.823294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef842b-892b-43a7-9bbe-5ecf77b63c4b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:25.863813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:25.898587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ae45f-821b-45c6-b3a1-ace7bd3b6d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:25.899241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ae45f-821b-45c6-b3a1-ace7bd3b6d0a service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:25.899528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ae45f-821b-45c6-b3a1-ace7bd3b6d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:25.899802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ae45f-821b-45c6-b3a1-ace7bd3b6d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:25.901484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-df6ae45f-821b-45c6-b3a1-ace7bd3b6d0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:26.011568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ae45f-821b-45c6-b3a1-ace7bd3b6d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:26.013065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ae45f-821b-45c6-b3a1-ace7bd3b6d0a service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:26.031338 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 720/1449] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:28.156211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623d4aa-dd8b-4e79-8874-c0d91be304e4 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:28.156765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623d4aa-dd8b-4e79-8874-c0d91be304e4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:28.156765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623d4aa-dd8b-4e79-8874-c0d91be304e4 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:28.158152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:28.163371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa176f5f-fcd7-4d4e-bb66-1c111ce4e8c3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:28.164336 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa176f5f-fcd7-4d4e-bb66-1c111ce4e8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:28.164442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa176f5f-fcd7-4d4e-bb66-1c111ce4e8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:28.230377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fa176f5f-fcd7-4d4e-bb66-1c111ce4e8c3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:28.231501 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fa176f5f-fcd7-4d4e-bb66-1c111ce4e8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:28.252278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:28.336500 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:28.375597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623d4aa-dd8b-4e79-8874-c0d91be304e4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:28.377200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1623d4aa-dd8b-4e79-8874-c0d91be304e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:28.482463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623d4aa-dd8b-4e79-8874-c0d91be304e4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:28.502701 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:28.556795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a920f8-bb5b-4a34-8c65-ed7f8ce0a18d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:28.557484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a920f8-bb5b-4a34-8c65-ed7f8ce0a18d service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:28.557773 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a920f8-bb5b-4a34-8c65-ed7f8ce0a18d service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:28.558053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a920f8-bb5b-4a34-8c65-ed7f8ce0a18d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:28.559744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a3a920f8-bb5b-4a34-8c65-ed7f8ce0a18d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:28.672913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a920f8-bb5b-4a34-8c65-ed7f8ce0a18d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:28.674439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a920f8-bb5b-4a34-8c65-ed7f8ce0a18d service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:28.692777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 731/1452] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:30.788685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e45b50-eb7d-4c05-a318-c6dbc50e27c6 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:30.789250 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e45b50-eb7d-4c05-a318-c6dbc50e27c6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:30.789410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e45b50-eb7d-4c05-a318-c6dbc50e27c6 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:30.790671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:30.795940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb6ea3-9811-41c1-b8ca-e9b1a7eefdd7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:30.796360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb6ea3-9811-41c1-b8ca-e9b1a7eefdd7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:30.796636 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb6ea3-9811-41c1-b8ca-e9b1a7eefdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:30.860775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:30.863513 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-86eb6ea3-9811-41c1-b8ca-e9b1a7eefdd7 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:30.864758 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-86eb6ea3-9811-41c1-b8ca-e9b1a7eefdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:30.963979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:31.004073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e45b50-eb7d-4c05-a318-c6dbc50e27c6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:31.005687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-45e45b50-eb7d-4c05-a318-c6dbc50e27c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:31.109997 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e45b50-eb7d-4c05-a318-c6dbc50e27c6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:31.159592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:31.185300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde2ae4-276f-48ba-b162-793f851c676f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:31.185949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2ae4-276f-48ba-b162-793f851c676f service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:31.186235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2ae4-276f-48ba-b162-793f851c676f service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:31.186512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde2ae4-276f-48ba-b162-793f851c676f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:31.188208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-afde2ae4-276f-48ba-b162-793f851c676f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:31.309296 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde2ae4-276f-48ba-b162-793f851c676f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:31.311076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde2ae4-276f-48ba-b162-793f851c676f service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:31.329840 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 723/1455] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:33.477938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40089fb2-8b38-469e-b736-79b0b5778172 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:33.478509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40089fb2-8b38-469e-b736-79b0b5778172 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:33.478509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40089fb2-8b38-469e-b736-79b0b5778172 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:33.479871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:33.486350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637da344-6bd4-43df-b77a-995c572b35e6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:33.486865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637da344-6bd4-43df-b77a-995c572b35e6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:33.486971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637da344-6bd4-43df-b77a-995c572b35e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:33.555237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-637da344-6bd4-43df-b77a-995c572b35e6 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:33.556468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-637da344-6bd4-43df-b77a-995c572b35e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:33.590048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:33.664822 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:33.704371 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40089fb2-8b38-469e-b736-79b0b5778172 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:33.705975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-40089fb2-8b38-469e-b736-79b0b5778172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:33.826437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40089fb2-8b38-469e-b736-79b0b5778172 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:33.854966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:33.903006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c5f7a-6832-44f8-be5d-e1ae60bdd159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:33.903006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c5f7a-6832-44f8-be5d-e1ae60bdd159 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:33.903006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c5f7a-6832-44f8-be5d-e1ae60bdd159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:33.903006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c5f7a-6832-44f8-be5d-e1ae60bdd159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:33.904145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-499c5f7a-6832-44f8-be5d-e1ae60bdd159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:34.022261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c5f7a-6832-44f8-be5d-e1ae60bdd159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:34.024176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c5f7a-6832-44f8-be5d-e1ae60bdd159 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:34.042527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 734/1458] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:36.251342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89f76e-e5a2-4374-9643-bb0c5e78aad4 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:36.251867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89f76e-e5a2-4374-9643-bb0c5e78aad4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:36.251967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89f76e-e5a2-4374-9643-bb0c5e78aad4 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:36.253346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:36.259409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9031f-39f8-4c34-b9c4-41362df08f80 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:36.259859 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9031f-39f8-4c34-b9c4-41362df08f80 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:36.259958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9031f-39f8-4c34-b9c4-41362df08f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:36.325183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-69f9031f-39f8-4c34-b9c4-41362df08f80 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:36.326408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-69f9031f-39f8-4c34-b9c4-41362df08f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:36.334743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:36.433979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:36.473189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89f76e-e5a2-4374-9643-bb0c5e78aad4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:36.474735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ab89f76e-e5a2-4374-9643-bb0c5e78aad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:36.579904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89f76e-e5a2-4374-9643-bb0c5e78aad4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:36.637396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:36.655801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d617862-2923-48e3-8775-412bf893f41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:36.656452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d617862-2923-48e3-8775-412bf893f41d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:36.656733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d617862-2923-48e3-8775-412bf893f41d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:36.657038 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d617862-2923-48e3-8775-412bf893f41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:36.658694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2d617862-2923-48e3-8775-412bf893f41d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:36.769378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d617862-2923-48e3-8775-412bf893f41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:36.770897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d617862-2923-48e3-8775-412bf893f41d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:36.790463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 726/1461] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:39.031616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b90e5-e080-4784-b7e2-18e574e5de84 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:39.032170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b90e5-e080-4784-b7e2-18e574e5de84 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:39.032170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b90e5-e080-4784-b7e2-18e574e5de84 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:39.033657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:39.039377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e415-e606-4026-ae05-f7dfdf153e7e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:39.039770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e415-e606-4026-ae05-f7dfdf153e7e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:39.039936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684e415-e606-4026-ae05-f7dfdf153e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:39.103966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1684e415-e606-4026-ae05-f7dfdf153e7e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:39.105224 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1684e415-e606-4026-ae05-f7dfdf153e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:39.142582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:39.202083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:39.239604 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b90e5-e080-4784-b7e2-18e574e5de84 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:39.241210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b99b90e5-e080-4784-b7e2-18e574e5de84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:39.366532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b90e5-e080-4784-b7e2-18e574e5de84 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:39.405014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:39.461494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b3687-d3bd-42d8-a7bc-d93f0e24ad56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:39.462150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b3687-d3bd-42d8-a7bc-d93f0e24ad56 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:39.462441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b3687-d3bd-42d8-a7bc-d93f0e24ad56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:39.462716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b3687-d3bd-42d8-a7bc-d93f0e24ad56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:39.464391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-683b3687-d3bd-42d8-a7bc-d93f0e24ad56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:39.577628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b3687-d3bd-42d8-a7bc-d93f0e24ad56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:39.579090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b3687-d3bd-42d8-a7bc-d93f0e24ad56 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:39.600837 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 737/1464] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:41.666045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0ef79-5f61-429c-aefa-82d01c69e1f8 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:41.666612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0ef79-5f61-429c-aefa-82d01c69e1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:41.666612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0ef79-5f61-429c-aefa-82d01c69e1f8 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:41.667987 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:41.673285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbb211-9071-45d6-a5d5-d3be13a98b55 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:41.673692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbb211-9071-45d6-a5d5-d3be13a98b55 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:41.673855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbb211-9071-45d6-a5d5-d3be13a98b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:41.744628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-24cbb211-9071-45d6-a5d5-d3be13a98b55 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:41.745889 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-24cbb211-9071-45d6-a5d5-d3be13a98b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:41.767193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:41.850758 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:41.896665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0ef79-5f61-429c-aefa-82d01c69e1f8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:41.900093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-25a0ef79-5f61-429c-aefa-82d01c69e1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:42.018595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0ef79-5f61-429c-aefa-82d01c69e1f8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:42.089000 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:42.097543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25daf6b-cd3f-4c66-b768-4a7388c28596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:42.102432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25daf6b-cd3f-4c66-b768-4a7388c28596 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:42.102432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25daf6b-cd3f-4c66-b768-4a7388c28596 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:42.102432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25daf6b-cd3f-4c66-b768-4a7388c28596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:42.102432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c25daf6b-cd3f-4c66-b768-4a7388c28596 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:42.238773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25daf6b-cd3f-4c66-b768-4a7388c28596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:42.240135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25daf6b-cd3f-4c66-b768-4a7388c28596 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:42.262499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 729/1467] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:44.390434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776238c-ae36-44c4-8a03-a1bb36336a2e None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:44.390950 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776238c-ae36-44c4-8a03-a1bb36336a2e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:44.391046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776238c-ae36-44c4-8a03-a1bb36336a2e None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:44.392446 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:43:44.398907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b122b95-c68e-4318-952a-65475586a494 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:44.399381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b122b95-c68e-4318-952a-65475586a494 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:44.399495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b122b95-c68e-4318-952a-65475586a494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:44.480790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9b122b95-c68e-4318-952a-65475586a494 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:43:44.482065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9b122b95-c68e-4318-952a-65475586a494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:44.535456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:44.585966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:43:44.623929 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776238c-ae36-44c4-8a03-a1bb36336a2e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:43:44.625502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6776238c-ae36-44c4-8a03-a1bb36336a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:44.732559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776238c-ae36-44c4-8a03-a1bb36336a2e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:43:44.783092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:44.809760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4682511-8bdd-40a4-a46f-1b4b641854eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:43:44.810412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4682511-8bdd-40a4-a46f-1b4b641854eb service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:43:44.810697 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4682511-8bdd-40a4-a46f-1b4b641854eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:43:44.810972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4682511-8bdd-40a4-a46f-1b4b641854eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:43:44.812633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b4682511-8bdd-40a4-a46f-1b4b641854eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:43:44.921920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4682511-8bdd-40a4-a46f-1b4b641854eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:43:44.923555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4682511-8bdd-40a4-a46f-1b4b641854eb service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:43:44.942814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 740/1470] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:43:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:43:54.583397 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=22 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:43:54.940396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=66 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:04.593347 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d43634-68e7-4112-82eb-61b213741a36 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:04.593915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d43634-68e7-4112-82eb-61b213741a36 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:04.593915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d43634-68e7-4112-82eb-61b213741a36 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:04.595377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 731/1471] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:04.601237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f74394-4b05-4136-99a4-1f78cde407e5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:04.601949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f74394-4b05-4136-99a4-1f78cde407e5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:04.601949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f74394-4b05-4136-99a4-1f78cde407e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:04.668301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-34f74394-4b05-4136-99a4-1f78cde407e5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:04.669566 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-34f74394-4b05-4136-99a4-1f78cde407e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:04.784956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 741/1472] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:04.834803 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea623b63-97b0-4819-aae2-141a8886e5e9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:04.835094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea623b63-97b0-4819-aae2-141a8886e5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:04.835398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea623b63-97b0-4819-aae2-141a8886e5e9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:04.836762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 732/1473] 10.222.0.88 () {54 vars in 961 bytes} [Mon Jul 18 16:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:04.841721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c07d-dd26-4261-8c34-329dfba4fce2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:04.842018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c07d-dd26-4261-8c34-329dfba4fce2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:04.842303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c07d-dd26-4261-8c34-329dfba4fce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:04.931731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8269c07d-dd26-4261-8c34-329dfba4fce2 None None] MFA Rules not processed for user `54c7869983bf4534b3e730af0185eac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:04.932941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8269c07d-dd26-4261-8c34-329dfba4fce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:04.956820 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:05.025586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 742/1474] 10.222.0.88 () {58 vars in 1058 bytes} [Mon Jul 18 16:44:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:05.033281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea623b63-97b0-4819-aae2-141a8886e5e9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:05.034770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ea623b63-97b0-4819-aae2-141a8886e5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:05.140838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea623b63-97b0-4819-aae2-141a8886e5e9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:05.217479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f8ef0-7b38-4dc7-bb59-9a6b3c3aff57 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54c7869983bf4534b3e730af0185eac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:05.218040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f8ef0-7b38-4dc7-bb59-9a6b3c3aff57 service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:05.218320 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f8ef0-7b38-4dc7-bb59-9a6b3c3aff57 service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:05.218599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f8ef0-7b38-4dc7-bb59-9a6b3c3aff57 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:05.220236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-706f8ef0-7b38-4dc7-bb59-9a6b3c3aff57 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:05.275842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:05.331436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f8ef0-7b38-4dc7-bb59-9a6b3c3aff57 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:05.332916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f8ef0-7b38-4dc7-bb59-9a6b3c3aff57 service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:05.353535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 733/1475] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:07.538573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c467d44-e5b5-432f-99bb-b111581a800d None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:07.539167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c467d44-e5b5-432f-99bb-b111581a800d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:07.539167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c467d44-e5b5-432f-99bb-b111581a800d None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:07.540651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 743/1476] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:07.547034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c4608-4099-414e-8709-7d244b635821 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:07.547559 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c4608-4099-414e-8709-7d244b635821 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:07.547674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c4608-4099-414e-8709-7d244b635821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:07.620160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-860c4608-4099-414e-8709-7d244b635821 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:07.621431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-860c4608-4099-414e-8709-7d244b635821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:07.722115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 734/1477] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:07.746260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c467d44-e5b5-432f-99bb-b111581a800d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:07.747847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6c467d44-e5b5-432f-99bb-b111581a800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:07.885491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c467d44-e5b5-432f-99bb-b111581a800d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:07.936683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:07.985317 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a9782-6c90-4e49-b6f4-4387be8292bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:07.985983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9782-6c90-4e49-b6f4-4387be8292bc admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:07.986270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9782-6c90-4e49-b6f4-4387be8292bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:07.986549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a9782-6c90-4e49-b6f4-4387be8292bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:07.987291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a9782-6c90-4e49-b6f4-4387be8292bc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:07.989164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-0d4a9782-6c90-4e49-b6f4-4387be8292bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:44:07.989164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 18 16:44:07.989164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: Jul 18 16:44:07.990350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a9782-6c90-4e49-b6f4-4387be8292bc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:08.012362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 744/1478] 10.222.0.88 () {56 vars in 1108 bytes} [Mon Jul 18 16:44:07 2022] GET /identity/v3/projects => generated 2553 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:44:10.058200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce4341-660a-4aed-b4c0-5af9e061647b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:10.058727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce4341-660a-4aed-b4c0-5af9e061647b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:10.058727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce4341-660a-4aed-b4c0-5af9e061647b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:10.060145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 735/1479] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:10.065490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5275ff-f9c4-48aa-8ebd-69d1922f2cc6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:10.065985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5275ff-f9c4-48aa-8ebd-69d1922f2cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:10.066098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5275ff-f9c4-48aa-8ebd-69d1922f2cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:10.130531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-de5275ff-f9c4-48aa-8ebd-69d1922f2cc6 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:10.131782 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-de5275ff-f9c4-48aa-8ebd-69d1922f2cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:10.351808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 745/1480] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:10.394270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce4341-660a-4aed-b4c0-5af9e061647b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:10.395798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-80ce4341-660a-4aed-b4c0-5af9e061647b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:10.406582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:10.551697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce4341-660a-4aed-b4c0-5af9e061647b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:10.635749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:10.670299 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15550704-ffd6-4a3b-a5a4-fcc8a5a0a1da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:10.670958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15550704-ffd6-4a3b-a5a4-fcc8a5a0a1da service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:10.671252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15550704-ffd6-4a3b-a5a4-fcc8a5a0a1da service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:10.671599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15550704-ffd6-4a3b-a5a4-fcc8a5a0a1da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:10.673213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-15550704-ffd6-4a3b-a5a4-fcc8a5a0a1da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:10.765357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15550704-ffd6-4a3b-a5a4-fcc8a5a0a1da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:10.766985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15550704-ffd6-4a3b-a5a4-fcc8a5a0a1da service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:10.785907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 736/1481] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:12.987070 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fbd61-196a-41f9-87b4-7fcf458c7873 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:12.987594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fbd61-196a-41f9-87b4-7fcf458c7873 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:12.987594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fbd61-196a-41f9-87b4-7fcf458c7873 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:12.988915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 746/1482] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:12.994941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43445be-6cce-4cfb-a89e-5f4d588e681a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:12.995442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43445be-6cce-4cfb-a89e-5f4d588e681a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:12.995555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43445be-6cce-4cfb-a89e-5f4d588e681a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:13.060509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c43445be-6cce-4cfb-a89e-5f4d588e681a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:13.061832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c43445be-6cce-4cfb-a89e-5f4d588e681a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:13.108895 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:13.159355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 737/1483] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:44:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:13.196577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fbd61-196a-41f9-87b4-7fcf458c7873 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:13.198201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-353fbd61-196a-41f9-87b4-7fcf458c7873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:13.222100 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:13.303004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fbd61-196a-41f9-87b4-7fcf458c7873 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:13.376581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362fdb6-5694-4238-b2d0-b33fbdbaf910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:13.377252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362fdb6-5694-4238-b2d0-b33fbdbaf910 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:13.377536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362fdb6-5694-4238-b2d0-b33fbdbaf910 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:13.377808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362fdb6-5694-4238-b2d0-b33fbdbaf910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:13.379462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4362fdb6-5694-4238-b2d0-b33fbdbaf910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:13.453485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:13.488602 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362fdb6-5694-4238-b2d0-b33fbdbaf910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:13.490083 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4362fdb6-5694-4238-b2d0-b33fbdbaf910 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:13.510007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 747/1484] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:15.591644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa3d7-0ec7-4be2-b8ff-e0a4d9ee2068 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:15.592229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa3d7-0ec7-4be2-b8ff-e0a4d9ee2068 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:15.592229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa3d7-0ec7-4be2-b8ff-e0a4d9ee2068 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:15.593679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 738/1485] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:15.599352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c047143-89c0-40a6-87e0-675c6fdce92a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:15.599857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c047143-89c0-40a6-87e0-675c6fdce92a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:15.599947 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c047143-89c0-40a6-87e0-675c6fdce92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:15.664779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3c047143-89c0-40a6-87e0-675c6fdce92a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:15.665945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3c047143-89c0-40a6-87e0-675c6fdce92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:15.766200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 748/1486] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:15.804102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa3d7-0ec7-4be2-b8ff-e0a4d9ee2068 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:15.805757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a29aa3d7-0ec7-4be2-b8ff-e0a4d9ee2068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:15.911604 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa3d7-0ec7-4be2-b8ff-e0a4d9ee2068 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:15.990000 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:15.992837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc8cd8-1e48-43e4-b795-0b3993472065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:15.993626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc8cd8-1e48-43e4-b795-0b3993472065 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:15.993912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc8cd8-1e48-43e4-b795-0b3993472065 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:15.994192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc8cd8-1e48-43e4-b795-0b3993472065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:15.995932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-29dc8cd8-1e48-43e4-b795-0b3993472065 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:16.107554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc8cd8-1e48-43e4-b795-0b3993472065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:16.109134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc8cd8-1e48-43e4-b795-0b3993472065 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:16.128308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 739/1487] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:18.305815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf962dc-725b-45f8-9be3-4a9adaf13c7a None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:18.306419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf962dc-725b-45f8-9be3-4a9adaf13c7a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:18.306419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf962dc-725b-45f8-9be3-4a9adaf13c7a None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:18.307743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 749/1488] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:18.313176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73786272-ccfa-4462-b487-8a3aa3c4e0b4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:18.313617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73786272-ccfa-4462-b487-8a3aa3c4e0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:18.313744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73786272-ccfa-4462-b487-8a3aa3c4e0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:18.380586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-73786272-ccfa-4462-b487-8a3aa3c4e0b4 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:18.382326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-73786272-ccfa-4462-b487-8a3aa3c4e0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:18.437126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:18.482257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 740/1489] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:18.523413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf962dc-725b-45f8-9be3-4a9adaf13c7a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:18.525006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cdf962dc-725b-45f8-9be3-4a9adaf13c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:18.561878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:18.631445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf962dc-725b-45f8-9be3-4a9adaf13c7a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:18.706110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15414c-6059-449b-8403-6864a8a59b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:18.706753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15414c-6059-449b-8403-6864a8a59b46 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:18.707038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15414c-6059-449b-8403-6864a8a59b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:18.707307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15414c-6059-449b-8403-6864a8a59b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:18.709000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ec15414c-6059-449b-8403-6864a8a59b46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:18.789081 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:18.819078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15414c-6059-449b-8403-6864a8a59b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:18.820454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15414c-6059-449b-8403-6864a8a59b46 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:18.838756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 750/1490] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:21.305124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3f26c-90ee-448d-89de-758856790884 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:21.305728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3f26c-90ee-448d-89de-758856790884 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:21.305728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3f26c-90ee-448d-89de-758856790884 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:21.307123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 741/1491] 10.222.0.88 () {54 vars in 926 bytes} [Mon Jul 18 16:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:44:21.313007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c4eb4-feab-4ea3-af85-1a85d28967ea None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:21.313494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c4eb4-feab-4ea3-af85-1a85d28967ea None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:21.313596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c4eb4-feab-4ea3-af85-1a85d28967ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:21.380183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2e0c4eb4-feab-4ea3-af85-1a85d28967ea None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:44:21.381969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2e0c4eb4-feab-4ea3-af85-1a85d28967ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:21.479920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 751/1492] 10.222.0.88 () {58 vars in 1023 bytes} [Mon Jul 18 16:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:44:21.519445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3f26c-90ee-448d-89de-758856790884 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:44:21.521052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-44a3f26c-90ee-448d-89de-758856790884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:21.625329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3f26c-90ee-448d-89de-758856790884 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:44:21.704150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544bea8-4ef9-42a3-96a6-e477725fc113 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:44:21.704807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544bea8-4ef9-42a3-96a6-e477725fc113 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:44:21.705117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544bea8-4ef9-42a3-96a6-e477725fc113 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:44:21.705391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544bea8-4ef9-42a3-96a6-e477725fc113 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:44:21.707042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d544bea8-4ef9-42a3-96a6-e477725fc113 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:44:21.716354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:21.816665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544bea8-4ef9-42a3-96a6-e477725fc113 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:44:21.818767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544bea8-4ef9-42a3-96a6-e477725fc113 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:44:21.836770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 742/1493] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:44:31.478066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=82 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:44:31.834485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=50 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:59.058234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449851a9-125e-4fd8-be9b-2cad176524b4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:59.058831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449851a9-125e-4fd8-be9b-2cad176524b4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:59.058831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449851a9-125e-4fd8-be9b-2cad176524b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:59.135226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-449851a9-125e-4fd8-be9b-2cad176524b4 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:59.137119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-449851a9-125e-4fd8-be9b-2cad176524b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:59.236069 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 752/1494] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:59.244098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e56c84-872a-4a06-9f5c-3979f740a2ba None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:59.244592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e56c84-872a-4a06-9f5c-3979f740a2ba None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:59.244703 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e56c84-872a-4a06-9f5c-3979f740a2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:59.319167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c2e56c84-872a-4a06-9f5c-3979f740a2ba None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:45:59.320403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c2e56c84-872a-4a06-9f5c-3979f740a2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:59.424953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 743/1495] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:45:59.429517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449851a9-125e-4fd8-be9b-2cad176524b4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:59.430848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-449851a9-125e-4fd8-be9b-2cad176524b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:59.442375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449851a9-125e-4fd8-be9b-2cad176524b4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:59.504172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:59.539171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bc866-294c-4b15-abda-6b5a645aec0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:59.539833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bc866-294c-4b15-abda-6b5a645aec0b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:59.540115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bc866-294c-4b15-abda-6b5a645aec0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:59.540390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bc866-294c-4b15-abda-6b5a645aec0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:59.541145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bc866-294c-4b15-abda-6b5a645aec0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:45:59.541444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bc866-294c-4b15-abda-6b5a645aec0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:59.542468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-b77bc866-294c-4b15-abda-6b5a645aec0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:45:59.542468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 18 16:45:59.542468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: Jul 18 16:45:59.543358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bc866-294c-4b15-abda-6b5a645aec0b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:59.554492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 753/1496] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:45:59.562741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e56c84-872a-4a06-9f5c-3979f740a2ba None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:59.572608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e56c84-872a-4a06-9f5c-3979f740a2ba None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:59.633807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:45:59.672744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb58ca-7ef5-40e0-996d-fecc4ed47966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:59.673479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb58ca-7ef5-40e0-996d-fecc4ed47966 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:59.673757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb58ca-7ef5-40e0-996d-fecc4ed47966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:59.674027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb58ca-7ef5-40e0-996d-fecc4ed47966 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:59.674727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb58ca-7ef5-40e0-996d-fecc4ed47966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:59.676455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-71eb58ca-7ef5-40e0-996d-fecc4ed47966 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:45:59.676455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 18 16:45:59.676455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: Jul 18 16:45:59.677668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb58ca-7ef5-40e0-996d-fecc4ed47966 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:45:59.808988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 744/1497] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:45:59 2022] POST /identity/v3/projects => generated 379 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:45:59.813268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bc866-294c-4b15-abda-6b5a645aec0b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:45:59.814831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b77bc866-294c-4b15-abda-6b5a645aec0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:45:59.850679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bc866-294c-4b15-abda-6b5a645aec0b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:45:59.948200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22d0dd-c678-43dd-89ab-a3ffb6835dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:45:59.948872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22d0dd-c678-43dd-89ab-a3ffb6835dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:45:59.949189 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22d0dd-c678-43dd-89ab-a3ffb6835dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:45:59.949465 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22d0dd-c678-43dd-89ab-a3ffb6835dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:45:59.950165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22d0dd-c678-43dd-89ab-a3ffb6835dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:45:59.951178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-aa22d0dd-c678-43dd-89ab-a3ffb6835dd9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:45:59.951178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 18 16:45:59.951178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: Jul 18 16:45:59.952383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22d0dd-c678-43dd-89ab-a3ffb6835dd9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:00.046738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 754/1498] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:45:59 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:46:00.050943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb58ca-7ef5-40e0-996d-fecc4ed47966 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:00.053115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-71eb58ca-7ef5-40e0-996d-fecc4ed47966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:00.115309 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb58ca-7ef5-40e0-996d-fecc4ed47966 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:00.199862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:00.211215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea679c7-84a8-4e52-b141-2b5a734da886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:00.219035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea679c7-84a8-4e52-b141-2b5a734da886 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:00.219035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea679c7-84a8-4e52-b141-2b5a734da886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:00.219035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea679c7-84a8-4e52-b141-2b5a734da886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:00.219035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea679c7-84a8-4e52-b141-2b5a734da886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:00.219035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-bea679c7-84a8-4e52-b141-2b5a734da886 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:46:00.219035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 18 16:46:00.219035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: Jul 18 16:46:00.219655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea679c7-84a8-4e52-b141-2b5a734da886 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:00.229093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 745/1499] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:00.234264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22d0dd-c678-43dd-89ab-a3ffb6835dd9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:00.234755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-aa22d0dd-c678-43dd-89ab-a3ffb6835dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:00.246501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22d0dd-c678-43dd-89ab-a3ffb6835dd9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:00.272614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:00.342706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420334a0-471e-49e8-a613-c0bfde200985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:00.343377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420334a0-471e-49e8-a613-c0bfde200985 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:00.343922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420334a0-471e-49e8-a613-c0bfde200985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:00.344212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420334a0-471e-49e8-a613-c0bfde200985 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:00.344954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420334a0-471e-49e8-a613-c0bfde200985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:00.346060 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-420334a0-471e-49e8-a613-c0bfde200985 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:46:00.346060 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 18 16:46:00.346060 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: Jul 18 16:46:00.346941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420334a0-471e-49e8-a613-c0bfde200985 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:00.359801 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 755/1500] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:46:00.364167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea679c7-84a8-4e52-b141-2b5a734da886 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:00.366376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bea679c7-84a8-4e52-b141-2b5a734da886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:00.399318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea679c7-84a8-4e52-b141-2b5a734da886 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:00.499988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9794f1a-5135-4303-aee4-884f327a9812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:00.500900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9794f1a-5135-4303-aee4-884f327a9812 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:00.501193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9794f1a-5135-4303-aee4-884f327a9812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:00.501474 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9794f1a-5135-4303-aee4-884f327a9812 admin admin] PATH_INFO: `/v3/projects/9e8fd43368b34f27a0d190d0ec880cb4/users/fdc02839659540fb8228c942631ad017/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:00.524344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9794f1a-5135-4303-aee4-884f327a9812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e8fd43368b34f27a0d190d0ec880cb4, user_id=fdc02839659540fb8228c942631ad017, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:00.525437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-d9794f1a-5135-4303-aee4-884f327a9812 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:46:00.525437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 18 16:46:00.525437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: Jul 18 16:46:00.527391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9794f1a-5135-4303-aee4-884f327a9812 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:00.614032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 746/1501] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:46:00 2022] PUT /identity/v3/projects/9e8fd43368b34f27a0d190d0ec880cb4/users/fdc02839659540fb8228c942631ad017/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:00.626585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a348f33-5ffc-41b2-8fa6-539876c51c5c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:00.626886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a348f33-5ffc-41b2-8fa6-539876c51c5c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:00.627211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a348f33-5ffc-41b2-8fa6-539876c51c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:00.687313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5a348f33-5ffc-41b2-8fa6-539876c51c5c None None] MFA Rules not processed for user `fdc02839659540fb8228c942631ad017`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:00.688549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5a348f33-5ffc-41b2-8fa6-539876c51c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:00.768504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 756/1502] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:46:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:46:00.782444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b029c-e2d9-499b-88fc-3a768553f36c None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:00.782745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b029c-e2d9-499b-88fc-3a768553f36c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:00.783032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b029c-e2d9-499b-88fc-3a768553f36c None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:00.784457 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 747/1503] 10.222.0.88 () {54 vars in 964 bytes} [Mon Jul 18 16:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:46:00.789907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e691d4f-355b-4161-ac3c-f5c1fa3ba3f9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:00.790200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e691d4f-355b-4161-ac3c-f5c1fa3ba3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:00.790480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e691d4f-355b-4161-ac3c-f5c1fa3ba3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:00.822397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:00.866979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9e691d4f-355b-4161-ac3c-f5c1fa3ba3f9 None None] MFA Rules not processed for user `809604baaed7486fb1020263e99e9c63`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:00.868200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9e691d4f-355b-4161-ac3c-f5c1fa3ba3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:00.946920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 757/1504] 10.222.0.88 () {58 vars in 1061 bytes} [Mon Jul 18 16:46:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:00.954778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b029c-e2d9-499b-88fc-3a768553f36c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:00.956558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-783b029c-e2d9-499b-88fc-3a768553f36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:01.009542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:01.072953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b029c-e2d9-499b-88fc-3a768553f36c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:01.147734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24ee50-349b-46c2-9ef9-8d10814cacf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:01.148379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24ee50-349b-46c2-9ef9-8d10814cacf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:01.148669 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24ee50-349b-46c2-9ef9-8d10814cacf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:01.148975 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24ee50-349b-46c2-9ef9-8d10814cacf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:01.150669 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7d24ee50-349b-46c2-9ef9-8d10814cacf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:01.259526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d24ee50-349b-46c2-9ef9-8d10814cacf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:01.260990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d24ee50-349b-46c2-9ef9-8d10814cacf0 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:01.264434 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:01.279133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 748/1505] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:01.321188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8140e-b194-48b5-9f11-55b17dda53c3 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:01.321607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8140e-b194-48b5-9f11-55b17dda53c3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:01.321759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8140e-b194-48b5-9f11-55b17dda53c3 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:01.323050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 758/1506] 10.222.0.88 () {52 vars in 829 bytes} [Mon Jul 18 16:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:46:01.326568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24ee50-349b-46c2-9ef9-8d10814cacf0 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:01.327792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7d24ee50-349b-46c2-9ef9-8d10814cacf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:01.427546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24ee50-349b-46c2-9ef9-8d10814cacf0 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:01.521995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:01.522790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:01.523073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:01.523348 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] PATH_INFO: `/v3/users/fdc02839659540fb8228c942631ad017` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:01.535670 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdc02839659540fb8228c942631ad017)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:01.537333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:46:01.537333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 18 16:46:01.537333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: Jul 18 16:46:01.538545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:01.548476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:01.710944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdc02839659540fb8228c942631ad017'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.730014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdc02839659540fb8228c942631ad017'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.741935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdc02839659540fb8228c942631ad017'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.789247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdc02839659540fb8228c942631ad017'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.789709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdc02839659540fb8228c942631ad017'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:01.795563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 749/1507] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:46:01 2022] DELETE /identity/v3/users/fdc02839659540fb8228c942631ad017 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:01.800086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb98e6-bc7e-4313-8f80-449eafae3055 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:01.803578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb98e6-bc7e-4313-8f80-449eafae3055 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:01.803578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb98e6-bc7e-4313-8f80-449eafae3055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:01.858084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c4eb98e6-bc7e-4313-8f80-449eafae3055 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:46:01.859339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c4eb98e6-bc7e-4313-8f80-449eafae3055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:01.936206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:01.949089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 759/1508] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:46:01.960188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:01.961455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:02.060861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b61e1-0579-4306-958c-7b02c6e5ae85 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:02.119051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:02.137829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6f8c7-476e-4386-a79e-b0b7cfbf833d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:02.138491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6f8c7-476e-4386-a79e-b0b7cfbf833d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:02.138781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6f8c7-476e-4386-a79e-b0b7cfbf833d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:02.139063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6f8c7-476e-4386-a79e-b0b7cfbf833d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:02.140714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e0f6f8c7-476e-4386-a79e-b0b7cfbf833d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:02.251271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6f8c7-476e-4386-a79e-b0b7cfbf833d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:02.252730 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6f8c7-476e-4386-a79e-b0b7cfbf833d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:02.271578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 750/1509] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:46:02.917648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb98e6-bc7e-4313-8f80-449eafae3055 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:46:02.918956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c4eb98e6-bc7e-4313-8f80-449eafae3055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:46:02.931028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb98e6-bc7e-4313-8f80-449eafae3055 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:46:03.039135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:46:03.039996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:46:03.040282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:46:03.040560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] PATH_INFO: `/v3/projects/9e8fd43368b34f27a0d190d0ec880cb4` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:46:03.050842 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e8fd43368b34f27a0d190d0ec880cb4)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:46:03.051943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:46:03.051943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 18 16:46:03.051943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: Jul 18 16:46:03.055652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:46:03.170633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e8fd43368b34f27a0d190d0ec880cb4', 'request_id': 'req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:03.185056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e8fd43368b34f27a0d190d0ec880cb4', 'request_id': 'req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:03.191462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e8fd43368b34f27a0d190d0ec880cb4', 'request_id': 'req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:03.274992 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] The token cache is being invalidate because project 9e8fd43368b34f27a0d190d0ec880cb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:46:03.275286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:03.275835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7f5ddb40-32bb-42af-8f42-b1901c1a1bdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:46:03.277270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 760/1510] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:46:02 2022] DELETE /identity/v3/projects/9e8fd43368b34f27a0d190d0ec880cb4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:46:12.269193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=62 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:46:13.272775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:51.980349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11836e-2f84-45b8-836c-07cb57196766 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.980964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11836e-2f84-45b8-836c-07cb57196766 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.980964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11836e-2f84-45b8-836c-07cb57196766 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:51.982325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:47:51.987436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf1996-798e-4626-9f14-91b86407a07c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:51.989589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf1996-798e-4626-9f14-91b86407a07c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:51.989589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf1996-798e-4626-9f14-91b86407a07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.070015 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-32bf1996-798e-4626-9f14-91b86407a07c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:52.071247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-32bf1996-798e-4626-9f14-91b86407a07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:52.198817 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:52.403212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11836e-2f84-45b8-836c-07cb57196766 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:52.406938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0d11836e-2f84-45b8-836c-07cb57196766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:52.539887 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11836e-2f84-45b8-836c-07cb57196766 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:52.545358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e175e-7336-4bd2-b33f-783eb1b4b36c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:52.546013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e175e-7336-4bd2-b33f-783eb1b4b36c service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:52.546289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e175e-7336-4bd2-b33f-783eb1b4b36c service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:52.546560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e175e-7336-4bd2-b33f-783eb1b4b36c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:52.556576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e175e-7336-4bd2-b33f-783eb1b4b36c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:52.558196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e175e-7336-4bd2-b33f-783eb1b4b36c service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:52.567722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 752/1513] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:53.402522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e855f1c-74b0-42f6-a449-99fc958d1cab None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.403094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e855f1c-74b0-42f6-a449-99fc958d1cab None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.403094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e855f1c-74b0-42f6-a449-99fc958d1cab None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.404476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:47:53.409682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0bb9-7d3a-4ca9-994a-6954d2a09a62 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.410137 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0bb9-7d3a-4ca9-994a-6954d2a09a62 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.410243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0bb9-7d3a-4ca9-994a-6954d2a09a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.428979 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-913b0bb9-7d3a-4ca9-994a-6954d2a09a62 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:53.430205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-913b0bb9-7d3a-4ca9-994a-6954d2a09a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:53.438299 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:47:53.608748 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e855f1c-74b0-42f6-a449-99fc958d1cab None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:47:53.623999 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e855f1c-74b0-42f6-a449-99fc958d1cab None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:47:53.630877 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab45b56-b0bb-4e39-aa7d-650475a691c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:47:53.631521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab45b56-b0bb-4e39-aa7d-650475a691c6 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.631807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab45b56-b0bb-4e39-aa7d-650475a691c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.632086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab45b56-b0bb-4e39-aa7d-650475a691c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.640723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab45b56-b0bb-4e39-aa7d-650475a691c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:47:53.642404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab45b56-b0bb-4e39-aa7d-650475a691c6 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:47:53.651985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 763/1516] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:47:53.938371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd260-f39b-4542-9249-b935527441bc None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.938842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd260-f39b-4542-9249-b935527441bc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.939265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dd260-f39b-4542-9249-b935527441bc None None] PATH_INFO: `/v3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:53.940745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 754/1517] 10.222.0.66 () {54 vars in 925 bytes} [Mon Jul 18 16:47:53 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:47:53.947229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3762b-4810-496c-a72e-1423001685cb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:47:53.947516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3762b-4810-496c-a72e-1423001685cb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:47:53.947798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3762b-4810-496c-a72e-1423001685cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:47:54.009137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-60d3762b-4810-496c-a72e-1423001685cb None None] MFA Rules not processed for user `22bd4dab1fcd4ad4a8be886646d89242`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:47:54.010349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-60d3762b-4810-496c-a72e-1423001685cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:47:54.027289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:47:54.082878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 764/1518] 10.222.0.66 () {58 vars in 1015 bytes} [Mon Jul 18 16:47:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:03.281906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13075e-3281-462f-be21-c25b4aabdc22 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:03.282516 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13075e-3281-462f-be21-c25b4aabdc22 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:03.282516 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13075e-3281-462f-be21-c25b4aabdc22 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:03.283909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 755/1519] 10.222.0.88 () {54 vars in 961 bytes} [Mon Jul 18 16:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:48:03.289555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a25837-879f-4e81-9a72-59221f84f21e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:03.290052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a25837-879f-4e81-9a72-59221f84f21e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:03.290162 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a25837-879f-4e81-9a72-59221f84f21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:03.309180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f6a25837-879f-4e81-9a72-59221f84f21e None None] MFA Rules not processed for user `22bd4dab1fcd4ad4a8be886646d89242`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:03.310398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f6a25837-879f-4e81-9a72-59221f84f21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:03.321058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 765/1520] 10.222.0.88 () {58 vars in 1058 bytes} [Mon Jul 18 16:48:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:03.328486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13075e-3281-462f-be21-c25b4aabdc22 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:03.342557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed13075e-3281-462f-be21-c25b4aabdc22 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:03.350104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e5dc3-6667-4e2c-a210-6812b66cc376 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:03.350466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5dc3-6667-4e2c-a210-6812b66cc376 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:03.350898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5dc3-6667-4e2c-a210-6812b66cc376 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:03.352148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5dc3-6667-4e2c-a210-6812b66cc376 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:03.362361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e5dc3-6667-4e2c-a210-6812b66cc376 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:03.363964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e5dc3-6667-4e2c-a210-6812b66cc376 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:03.374412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 756/1521] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:48:04.975098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e774a-ba37-4be2-9a4d-a1ff60476f98 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:04.975672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e774a-ba37-4be2-9a4d-a1ff60476f98 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:04.976030 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e774a-ba37-4be2-9a4d-a1ff60476f98 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:04.977609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:48:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:48:04.982904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed462cb-2fca-4d66-a030-dcad59ba1f39 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:04.983382 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed462cb-2fca-4d66-a030-dcad59ba1f39 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:04.983496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed462cb-2fca-4d66-a030-dcad59ba1f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:05.002160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8ed462cb-2fca-4d66-a030-dcad59ba1f39 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:05.003394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8ed462cb-2fca-4d66-a030-dcad59ba1f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:05.012294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:05.221210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e774a-ba37-4be2-9a4d-a1ff60476f98 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:05.240082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e774a-ba37-4be2-9a4d-a1ff60476f98 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:05.245691 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d4e63-0a74-4148-90d3-465d1c92b56a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:05.246335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d4e63-0a74-4148-90d3-465d1c92b56a service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:05.246622 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d4e63-0a74-4148-90d3-465d1c92b56a service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:05.246890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d4e63-0a74-4148-90d3-465d1c92b56a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:05.255736 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d4e63-0a74-4148-90d3-465d1c92b56a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:05.257861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6d4e63-0a74-4148-90d3-465d1c92b56a service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:05.267821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 767/1524] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:06.135421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04cbed-0b92-4da0-a623-fe3f71428a16 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.135958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04cbed-0b92-4da0-a623-fe3f71428a16 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.135958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04cbed-0b92-4da0-a623-fe3f71428a16 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.137426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 16:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:48:06.142559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d6b8d-a5f7-43ec-b583-0805211912b8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.143002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d6b8d-a5f7-43ec-b583-0805211912b8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.143142 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d6b8d-a5f7-43ec-b583-0805211912b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.165578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-626d6b8d-a5f7-43ec-b583-0805211912b8 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:06.166887 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-626d6b8d-a5f7-43ec-b583-0805211912b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:06.174899 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 16:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:06.329656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04cbed-0b92-4da0-a623-fe3f71428a16 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:06.345630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04cbed-0b92-4da0-a623-fe3f71428a16 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:06.352576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b8478-c514-4417-a60f-3c75f177614c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:06.353558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b8478-c514-4417-a60f-3c75f177614c service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:06.353684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b8478-c514-4417-a60f-3c75f177614c service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:06.353900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b8478-c514-4417-a60f-3c75f177614c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:06.363070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b8478-c514-4417-a60f-3c75f177614c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:06.364765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b8478-c514-4417-a60f-3c75f177614c service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:06.378172 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 759/1527] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:10.362002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541a237-e1d9-4e3f-a9bc-d5548f318ccc None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:10.362652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541a237-e1d9-4e3f-a9bc-d5548f318ccc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:10.362652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541a237-e1d9-4e3f-a9bc-d5548f318ccc None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:10.364200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:48:10.370148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659991fb-590c-43bc-9a72-5f08b22ee058 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:10.370628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659991fb-590c-43bc-9a72-5f08b22ee058 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:10.370730 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659991fb-590c-43bc-9a72-5f08b22ee058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:10.389611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-659991fb-590c-43bc-9a72-5f08b22ee058 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:10.390814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-659991fb-590c-43bc-9a72-5f08b22ee058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:10.399176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:10.442053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541a237-e1d9-4e3f-a9bc-d5548f318ccc None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:10.443806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a541a237-e1d9-4e3f-a9bc-d5548f318ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:10.528032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a541a237-e1d9-4e3f-a9bc-d5548f318ccc None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:10.531943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324b5d1-8df9-41cc-aeb1-7515d9a3c5e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:10.532559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324b5d1-8df9-41cc-aeb1-7515d9a3c5e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:10.532842 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324b5d1-8df9-41cc-aeb1-7515d9a3c5e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:10.533158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324b5d1-8df9-41cc-aeb1-7515d9a3c5e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:10.543651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324b5d1-8df9-41cc-aeb1-7515d9a3c5e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:10.545131 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324b5d1-8df9-41cc-aeb1-7515d9a3c5e5 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:10.555156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 770/1530] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:12.620681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd1285-c6cc-4169-9c12-d6d6e05cb1a9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:12.621282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd1285-c6cc-4169-9c12-d6d6e05cb1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:12.621282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd1285-c6cc-4169-9c12-d6d6e05cb1a9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:12.622724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:48:12.628573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2592a-11ac-4231-b169-223a59aab5f3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:12.629089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2592a-11ac-4231-b169-223a59aab5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:12.629205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2592a-11ac-4231-b169-223a59aab5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:12.651496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-eae2592a-11ac-4231-b169-223a59aab5f3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:12.652779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-eae2592a-11ac-4231-b169-223a59aab5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:12.663428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:12.707709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd1285-c6cc-4169-9c12-d6d6e05cb1a9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:12.719755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd1285-c6cc-4169-9c12-d6d6e05cb1a9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:12.723927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def519f-753a-47e0-857c-d8ea324ce8a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:12.724572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def519f-753a-47e0-857c-d8ea324ce8a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:12.724973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def519f-753a-47e0-857c-d8ea324ce8a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:12.725249 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def519f-753a-47e0-857c-d8ea324ce8a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:12.734986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def519f-753a-47e0-857c-d8ea324ce8a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:12.736541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def519f-753a-47e0-857c-d8ea324ce8a7 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:12.746517 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 762/1533] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:16.378034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d150070-1ba9-40cb-98e6-5324d6df6a74 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:16.378599 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d150070-1ba9-40cb-98e6-5324d6df6a74 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:16.378599 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d150070-1ba9-40cb-98e6-5324d6df6a74 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:16.379962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:48:16.385263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b041621-2641-42d6-8d2c-761892f62238 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:16.385678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b041621-2641-42d6-8d2c-761892f62238 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:16.385844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b041621-2641-42d6-8d2c-761892f62238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:16.403817 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9b041621-2641-42d6-8d2c-761892f62238 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:16.405708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9b041621-2641-42d6-8d2c-761892f62238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:16.414183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:16.715660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d150070-1ba9-40cb-98e6-5324d6df6a74 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:16.717623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3d150070-1ba9-40cb-98e6-5324d6df6a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:16.784893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:16.799226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d150070-1ba9-40cb-98e6-5324d6df6a74 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:16.807535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a6b54-0f14-4d94-85df-779f4636f127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:16.807535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a6b54-0f14-4d94-85df-779f4636f127 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:16.807535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a6b54-0f14-4d94-85df-779f4636f127 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:16.807535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a6b54-0f14-4d94-85df-779f4636f127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:16.813212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a6b54-0f14-4d94-85df-779f4636f127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:16.814660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a6b54-0f14-4d94-85df-779f4636f127 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:16.824612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 773/1536] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:18.883463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9322556-3507-4545-9902-d734a9abbf7d None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:18.884001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9322556-3507-4545-9902-d734a9abbf7d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:18.884001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9322556-3507-4545-9902-d734a9abbf7d None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:18.885415 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 16:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:48:18.891306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93245f3b-28e4-45c1-8654-0cab75876aa5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:18.891879 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93245f3b-28e4-45c1-8654-0cab75876aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:18.892038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93245f3b-28e4-45c1-8654-0cab75876aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:18.911141 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-93245f3b-28e4-45c1-8654-0cab75876aa5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:48:18.913004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-93245f3b-28e4-45c1-8654-0cab75876aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:18.920847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 16:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:48:19.226083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9322556-3507-4545-9902-d734a9abbf7d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:48:19.227786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c9322556-3507-4545-9902-d734a9abbf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:48:19.240690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9322556-3507-4545-9902-d734a9abbf7d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:48:19.244391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e9c69-287a-484f-9268-381d9532f590 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:48:19.245072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e9c69-287a-484f-9268-381d9532f590 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:48:19.245363 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e9c69-287a-484f-9268-381d9532f590 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:48:19.245643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e9c69-287a-484f-9268-381d9532f590 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:48:19.254819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e9c69-287a-484f-9268-381d9532f590 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:48:19.256804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e9c69-287a-484f-9268-381d9532f590 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:48:19.266440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 765/1539] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:48:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:48:28.906313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=11 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:48:29.262225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:53.050708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f57364-301f-49d7-838a-48d5036f06e9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.051827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f57364-301f-49d7-838a-48d5036f06e9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.052431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f57364-301f-49d7-838a-48d5036f06e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.081943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-60f57364-301f-49d7-838a-48d5036f06e9 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:53.084178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-60f57364-301f-49d7-838a-48d5036f06e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:53.093215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 775/1540] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:53.102334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4762ea-25ed-4e81-b8b5-1ca30bd96131 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.102904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4762ea-25ed-4e81-b8b5-1ca30bd96131 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.102904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4762ea-25ed-4e81-b8b5-1ca30bd96131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.133941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4d4762ea-25ed-4e81-b8b5-1ca30bd96131 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:53.135453 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4d4762ea-25ed-4e81-b8b5-1ca30bd96131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:53.149438 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 766/1541] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:53.154246 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f57364-301f-49d7-838a-48d5036f06e9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:53.170345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f57364-301f-49d7-838a-48d5036f06e9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:53.180128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc6b7c-44a7-404f-83d8-13686c251f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:53.181251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc6b7c-44a7-404f-83d8-13686c251f3f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.181755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc6b7c-44a7-404f-83d8-13686c251f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.182251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc6b7c-44a7-404f-83d8-13686c251f3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.183186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc6b7c-44a7-404f-83d8-13686c251f3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:49:53.183706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc6b7c-44a7-404f-83d8-13686c251f3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:53.185580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc6b7c-44a7-404f-83d8-13686c251f3f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:53.204567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 776/1542] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:49:53.214078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4762ea-25ed-4e81-b8b5-1ca30bd96131 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:53.231179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4762ea-25ed-4e81-b8b5-1ca30bd96131 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:53.237714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c470253-0ab3-40fb-aef1-0ab25d80278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:53.238395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c470253-0ab3-40fb-aef1-0ab25d80278b admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.238684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c470253-0ab3-40fb-aef1-0ab25d80278b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.238969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c470253-0ab3-40fb-aef1-0ab25d80278b admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.239741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c470253-0ab3-40fb-aef1-0ab25d80278b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:53.241774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c470253-0ab3-40fb-aef1-0ab25d80278b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:53.469315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 767/1543] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/projects => generated 375 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:53.475340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc6b7c-44a7-404f-83d8-13686c251f3f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:53.491305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc6b7c-44a7-404f-83d8-13686c251f3f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:53.491879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74298a4a-a2a7-4bd4-b9b4-58261a6bc83b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.492573 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74298a4a-a2a7-4bd4-b9b4-58261a6bc83b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.493118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74298a4a-a2a7-4bd4-b9b4-58261a6bc83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.526548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-74298a4a-a2a7-4bd4-b9b4-58261a6bc83b None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:53.528626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-74298a4a-a2a7-4bd4-b9b4-58261a6bc83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:53.629353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf4fd9-b295-452e-b84b-803823368682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:53.629927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 768/1544] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:53.631409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4fd9-b295-452e-b84b-803823368682 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.631912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4fd9-b295-452e-b84b-803823368682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.632413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4fd9-b295-452e-b84b-803823368682 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.633356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf4fd9-b295-452e-b84b-803823368682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:53.635529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf4fd9-b295-452e-b84b-803823368682 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:53.640033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d78a26-ab8f-4175-ac9f-a08834efef0c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.640033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d78a26-ab8f-4175-ac9f-a08834efef0c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.640033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d78a26-ab8f-4175-ac9f-a08834efef0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.679828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-25d78a26-ab8f-4175-ac9f-a08834efef0c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:53.683367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-25d78a26-ab8f-4175-ac9f-a08834efef0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:53.694869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 769/1545] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:49:53.699999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d78a26-ab8f-4175-ac9f-a08834efef0c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:53.713444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d78a26-ab8f-4175-ac9f-a08834efef0c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:53.725807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26354b7-fea8-4bf8-bcdb-fd9868895abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:53.726829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26354b7-fea8-4bf8-bcdb-fd9868895abf admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.727540 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26354b7-fea8-4bf8-bcdb-fd9868895abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.728629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26354b7-fea8-4bf8-bcdb-fd9868895abf admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.731308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 777/1546] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/users => generated 288 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:53.732332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26354b7-fea8-4bf8-bcdb-fd9868895abf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:49:53.732868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26354b7-fea8-4bf8-bcdb-fd9868895abf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:53.738392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-a26354b7-fea8-4bf8-bcdb-fd9868895abf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:49:53.738392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 18 16:49:53.738392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: Jul 18 16:49:53.740278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf4fd9-b295-452e-b84b-803823368682 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:53.741443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26354b7-fea8-4bf8-bcdb-fd9868895abf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:53.764335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 770/1547] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:49:53.766956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf4fd9-b295-452e-b84b-803823368682 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:53.770274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a93dd-65ad-4c51-b0ea-d8e831cb3443 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.771174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a93dd-65ad-4c51-b0ea-d8e831cb3443 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.772153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a93dd-65ad-4c51-b0ea-d8e831cb3443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.776679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac3be4-d8a5-4a13-9c7f-7c77c028f4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:53.781889 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac3be4-d8a5-4a13-9c7f-7c77c028f4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.783612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac3be4-d8a5-4a13-9c7f-7c77c028f4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.784125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac3be4-d8a5-4a13-9c7f-7c77c028f4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.785970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac3be4-d8a5-4a13-9c7f-7c77c028f4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:53.789928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac3be4-d8a5-4a13-9c7f-7c77c028f4d9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:53.814101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 778/1548] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:53.815308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac3be4-d8a5-4a13-9c7f-7c77c028f4d9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:53.816530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ea9a93dd-65ad-4c51-b0ea-d8e831cb3443 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:53.818491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ea9a93dd-65ad-4c51-b0ea-d8e831cb3443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:53.834161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 771/1549] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:53.835980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794c2fe-a793-49ee-bff3-722634ad1c7e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.838469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac3be4-d8a5-4a13-9c7f-7c77c028f4d9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:53.840388 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794c2fe-a793-49ee-bff3-722634ad1c7e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.841344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794c2fe-a793-49ee-bff3-722634ad1c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.848437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548b4f6-dd9d-4f5d-948b-53d6bfc8cd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:53.849972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548b4f6-dd9d-4f5d-948b-53d6bfc8cd18 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.852726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548b4f6-dd9d-4f5d-948b-53d6bfc8cd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.853276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548b4f6-dd9d-4f5d-948b-53d6bfc8cd18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.854210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548b4f6-dd9d-4f5d-948b-53d6bfc8cd18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:53.857270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-b548b4f6-dd9d-4f5d-948b-53d6bfc8cd18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:49:53.857270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 18 16:49:53.857270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: Jul 18 16:49:53.858832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548b4f6-dd9d-4f5d-948b-53d6bfc8cd18 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:53.874074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5794c2fe-a793-49ee-bff3-722634ad1c7e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:53.879444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5794c2fe-a793-49ee-bff3-722634ad1c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:53.894143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 772/1550] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:53.895398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794c2fe-a793-49ee-bff3-722634ad1c7e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:53.905884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794c2fe-a793-49ee-bff3-722634ad1c7e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:53.911254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1b3a4-7d07-4f25-940e-113653d83b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:53.912767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1b3a4-7d07-4f25-940e-113653d83b2f admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.913280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1b3a4-7d07-4f25-940e-113653d83b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.913899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1b3a4-7d07-4f25-940e-113653d83b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.914902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1b3a4-7d07-4f25-940e-113653d83b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:53.917091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1b3a4-7d07-4f25-940e-113653d83b2f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:53.930507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 773/1551] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:53.932351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05889ed-7d72-4c63-814a-fd6b06cf91fc None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.932632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05889ed-7d72-4c63-814a-fd6b06cf91fc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.933033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05889ed-7d72-4c63-814a-fd6b06cf91fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.952280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e05889ed-7d72-4c63-814a-fd6b06cf91fc None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:53.953547 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e05889ed-7d72-4c63-814a-fd6b06cf91fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:53.961684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 774/1552] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:53.964535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1529796-410a-4648-a704-a7452405ebd8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.964535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1529796-410a-4648-a704-a7452405ebd8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.964535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1529796-410a-4648-a704-a7452405ebd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.983279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e1529796-410a-4648-a704-a7452405ebd8 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:53.984422 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e1529796-410a-4648-a704-a7452405ebd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:53.993466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 779/1553] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/projects => generated 391 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:53.994811 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cc72d-9c20-4d57-bafa-2430d0c10e07 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:53.995310 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cc72d-9c20-4d57-bafa-2430d0c10e07 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:53.995656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cc72d-9c20-4d57-bafa-2430d0c10e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:53.999464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 775/1554] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:54.001166 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29c4f4-699f-4ae8-bbe8-c3abf84043af None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.001421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29c4f4-699f-4ae8-bbe8-c3abf84043af None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.001748 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29c4f4-699f-4ae8-bbe8-c3abf84043af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.015176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c65cc72d-9c20-4d57-bafa-2430d0c10e07 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:54.016484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c65cc72d-9c20-4d57-bafa-2430d0c10e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:54.020017 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ee29c4f4-699f-4ae8-bbe8-c3abf84043af None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:54.035844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ee29c4f4-699f-4ae8-bbe8-c3abf84043af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:54.079125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 780/1555] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:54.080517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa4967-58c0-4c73-a92b-c381bf5423fe None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.080818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa4967-58c0-4c73-a92b-c381bf5423fe None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.081139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa4967-58c0-4c73-a92b-c381bf5423fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.086686 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 776/1556] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:54.091317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29c4f4-699f-4ae8-bbe8-c3abf84043af None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.102276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29c4f4-699f-4ae8-bbe8-c3abf84043af None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.103632 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6eaa4967-58c0-4c73-a92b-c381bf5423fe None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:54.104868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6eaa4967-58c0-4c73-a92b-c381bf5423fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:54.107013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52cb35-1efb-497b-9bef-58a45fafbf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.107962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52cb35-1efb-497b-9bef-58a45fafbf6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.108387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52cb35-1efb-497b-9bef-58a45fafbf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.108667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52cb35-1efb-497b-9bef-58a45fafbf6f admin admin] PATH_INFO: `/v3/projects/67975f78146c41a9aecc4f09f993f6bc/users/4f6db5c6b0b64503afc740f5c4748ad1/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.111246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:54.114191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 781/1557] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:54.115581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa4967-58c0-4c73-a92b-c381bf5423fe None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.124425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52cb35-1efb-497b-9bef-58a45fafbf6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=67975f78146c41a9aecc4f09f993f6bc, user_id=4f6db5c6b0b64503afc740f5c4748ad1, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.129853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52cb35-1efb-497b-9bef-58a45fafbf6f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.129995 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa4967-58c0-4c73-a92b-c381bf5423fe None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.134422 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c484e-edbe-443d-8cd7-ede4744cb279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.135086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c484e-edbe-443d-8cd7-ede4744cb279 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.135367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c484e-edbe-443d-8cd7-ede4744cb279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.135643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c484e-edbe-443d-8cd7-ede4744cb279 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.136409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c484e-edbe-443d-8cd7-ede4744cb279 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:49:54.136708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c484e-edbe-443d-8cd7-ede4744cb279 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.138368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c484e-edbe-443d-8cd7-ede4744cb279 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.150666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 782/1558] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:49:54.152937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c484e-edbe-443d-8cd7-ede4744cb279 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.162100 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c484e-edbe-443d-8cd7-ede4744cb279 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.166768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefb5f5-0a0d-46ee-83e7-515d73f15923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.167444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefb5f5-0a0d-46ee-83e7-515d73f15923 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.167723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefb5f5-0a0d-46ee-83e7-515d73f15923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.167995 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefb5f5-0a0d-46ee-83e7-515d73f15923 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.168675 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefb5f5-0a0d-46ee-83e7-515d73f15923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.170662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefb5f5-0a0d-46ee-83e7-515d73f15923 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.206276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 777/1559] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:54 2022] PUT /identity/v3/projects/67975f78146c41a9aecc4f09f993f6bc/users/4f6db5c6b0b64503afc740f5c4748ad1/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:54.207770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480ea55-fa10-42b7-b11c-3175216fc9c1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.208061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480ea55-fa10-42b7-b11c-3175216fc9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.208942 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480ea55-fa10-42b7-b11c-3175216fc9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.227952 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3480ea55-fa10-42b7-b11c-3175216fc9c1 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:54.229191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3480ea55-fa10-42b7-b11c-3175216fc9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:54.282195 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:54.288138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 783/1560] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:54.293106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902895b-de96-43ef-b642-b4f0f37e2383 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.293106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902895b-de96-43ef-b642-b4f0f37e2383 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.293106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902895b-de96-43ef-b642-b4f0f37e2383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.293576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 778/1561] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:54.293576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480ea55-fa10-42b7-b11c-3175216fc9c1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.303493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480ea55-fa10-42b7-b11c-3175216fc9c1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.308440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd089ec9-b6c1-494e-b5c5-fd72a16e1b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.309127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd089ec9-b6c1-494e-b5c5-fd72a16e1b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.309421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd089ec9-b6c1-494e-b5c5-fd72a16e1b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.309700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd089ec9-b6c1-494e-b5c5-fd72a16e1b31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.310444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd089ec9-b6c1-494e-b5c5-fd72a16e1b31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:49:54.310571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f902895b-de96-43ef-b642-b4f0f37e2383 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:54.310739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd089ec9-b6c1-494e-b5c5-fd72a16e1b31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.311792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f902895b-de96-43ef-b642-b4f0f37e2383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:54.312394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd089ec9-b6c1-494e-b5c5-fd72a16e1b31 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.320010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 784/1562] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:54.321300 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ed826-f69c-4fef-8018-699f4cab8f9a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.321587 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ed826-f69c-4fef-8018-699f4cab8f9a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.321865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ed826-f69c-4fef-8018-699f4cab8f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.327078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 779/1563] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:49:54.328575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b33ff-bbf4-4360-ac16-4bebba839130 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.329095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b33ff-bbf4-4360-ac16-4bebba839130 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.329443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b33ff-bbf4-4360-ac16-4bebba839130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.343154 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9c1ed826-f69c-4fef-8018-699f4cab8f9a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:54.344611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9c1ed826-f69c-4fef-8018-699f4cab8f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:54.348236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-689b33ff-bbf4-4360-ac16-4bebba839130 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:54.350219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-689b33ff-bbf4-4360-ac16-4bebba839130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:54.353897 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 785/1564] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:54.354849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ed826-f69c-4fef-8018-699f4cab8f9a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.362737 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 780/1565] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:54.364079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f16c39-cf0d-4bdb-98c4-cf20d04e532c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.364468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f16c39-cf0d-4bdb-98c4-cf20d04e532c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.364745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f16c39-cf0d-4bdb-98c4-cf20d04e532c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.365047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ed826-f69c-4fef-8018-699f4cab8f9a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.370003 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aa5b6-902c-4146-88bb-f8bb41cb33db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.370653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aa5b6-902c-4146-88bb-f8bb41cb33db admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.370937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aa5b6-902c-4146-88bb-f8bb41cb33db admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.371214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aa5b6-902c-4146-88bb-f8bb41cb33db admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.371909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026aa5b6-902c-4146-88bb-f8bb41cb33db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.373865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026aa5b6-902c-4146-88bb-f8bb41cb33db admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.391843 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-99f16c39-cf0d-4bdb-98c4-cf20d04e532c None None] MFA Rules not processed for user `4f6db5c6b0b64503afc740f5c4748ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:54.393084 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-99f16c39-cf0d-4bdb-98c4-cf20d04e532c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:54.459442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 786/1566] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:54.460364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aa5b6-902c-4146-88bb-f8bb41cb33db admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.464985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 781/1567] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:54.470502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceb4bb-9813-41d1-8248-9065ed5efeb2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.470848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceb4bb-9813-41d1-8248-9065ed5efeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.471135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceb4bb-9813-41d1-8248-9065ed5efeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.499871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ccceb4bb-9813-41d1-8248-9065ed5efeb2 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:54.501196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ccceb4bb-9813-41d1-8248-9065ed5efeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:54.534266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aa5b6-902c-4146-88bb-f8bb41cb33db admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.538997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c147b20-4a22-4999-a553-06e491edbf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.539663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c147b20-4a22-4999-a553-06e491edbf38 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.539943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c147b20-4a22-4999-a553-06e491edbf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.540214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c147b20-4a22-4999-a553-06e491edbf38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.540965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c147b20-4a22-4999-a553-06e491edbf38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.542664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c147b20-4a22-4999-a553-06e491edbf38 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.551745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:54.556738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 787/1568] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:54.556738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c147b20-4a22-4999-a553-06e491edbf38 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.566265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c147b20-4a22-4999-a553-06e491edbf38 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.570266 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 782/1569] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:54.571159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccceb4bb-9813-41d1-8248-9065ed5efeb2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.572024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b8f9c-3346-4238-96fd-c6bad9f15ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.572695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b8f9c-3346-4238-96fd-c6bad9f15ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.573027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b8f9c-3346-4238-96fd-c6bad9f15ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.573312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b8f9c-3346-4238-96fd-c6bad9f15ba9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.574049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b8f9c-3346-4238-96fd-c6bad9f15ba9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:49:54.574359 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b8f9c-3346-4238-96fd-c6bad9f15ba9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.576569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b8f9c-3346-4238-96fd-c6bad9f15ba9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.582609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccceb4bb-9813-41d1-8248-9065ed5efeb2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.589870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9443d77-c034-4422-9383-fa628e8aea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.590587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9443d77-c034-4422-9383-fa628e8aea71 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.590703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 788/1570] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:49:54.590870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9443d77-c034-4422-9383-fa628e8aea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.591146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9443d77-c034-4422-9383-fa628e8aea71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.591533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b8f9c-3346-4238-96fd-c6bad9f15ba9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.591839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9443d77-c034-4422-9383-fa628e8aea71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.593773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9443d77-c034-4422-9383-fa628e8aea71 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.604476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b8f9c-3346-4238-96fd-c6bad9f15ba9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.609667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcafda-cb10-42f3-92bf-367cae92761c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.610180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcafda-cb10-42f3-92bf-367cae92761c admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.610555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcafda-cb10-42f3-92bf-367cae92761c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.610742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcafda-cb10-42f3-92bf-367cae92761c admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.611540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddcafda-cb10-42f3-92bf-367cae92761c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:49:54.611781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddcafda-cb10-42f3-92bf-367cae92761c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.613426 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddcafda-cb10-42f3-92bf-367cae92761c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.626512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 789/1571] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:49:54.627369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcafda-cb10-42f3-92bf-367cae92761c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.637095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcafda-cb10-42f3-92bf-367cae92761c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.642041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79218108-afa1-448d-bc57-9f16d77b9a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.642702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79218108-afa1-448d-bc57-9f16d77b9a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.642981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79218108-afa1-448d-bc57-9f16d77b9a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.643255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79218108-afa1-448d-bc57-9f16d77b9a04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.644040 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79218108-afa1-448d-bc57-9f16d77b9a04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:49:54.644352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79218108-afa1-448d-bc57-9f16d77b9a04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.646006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79218108-afa1-448d-bc57-9f16d77b9a04 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.658421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 790/1572] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:49:54.659214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79218108-afa1-448d-bc57-9f16d77b9a04 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.670232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79218108-afa1-448d-bc57-9f16d77b9a04 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.675176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa879b1-5bdb-4080-9466-972b735b1cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.675847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa879b1-5bdb-4080-9466-972b735b1cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.676125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa879b1-5bdb-4080-9466-972b735b1cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.676395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa879b1-5bdb-4080-9466-972b735b1cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.677707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa879b1-5bdb-4080-9466-972b735b1cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.679633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa879b1-5bdb-4080-9466-972b735b1cb0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.683743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 783/1573] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:54.686353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d393d0-b6a7-411f-81cd-356660c441c7 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.686353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d393d0-b6a7-411f-81cd-356660c441c7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.686353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d393d0-b6a7-411f-81cd-356660c441c7 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.688838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 784/1574] 10.222.0.88 () {54 vars in 962 bytes} [Mon Jul 18 16:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:49:54.689655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d393d0-b6a7-411f-81cd-356660c441c7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.758005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d393d0-b6a7-411f-81cd-356660c441c7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.762671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc966c-63b3-4fd0-b5d1-53a08390c9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.763351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc966c-63b3-4fd0-b5d1-53a08390c9aa admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.763766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc966c-63b3-4fd0-b5d1-53a08390c9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.764106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc966c-63b3-4fd0-b5d1-53a08390c9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.764853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc966c-63b3-4fd0-b5d1-53a08390c9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.767545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc966c-63b3-4fd0-b5d1-53a08390c9aa admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.777466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 791/1575] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:54.778355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa879b1-5bdb-4080-9466-972b735b1cb0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.783442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 785/1576] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:54.783442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc966c-63b3-4fd0-b5d1-53a08390c9aa admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.789523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa879b1-5bdb-4080-9466-972b735b1cb0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.793562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc966c-63b3-4fd0-b5d1-53a08390c9aa admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.796378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0195716-5f93-454d-9878-3287715614c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.797112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0195716-5f93-454d-9878-3287715614c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.797400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0195716-5f93-454d-9878-3287715614c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.797681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0195716-5f93-454d-9878-3287715614c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.798180 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55e2f7-5c3e-40b5-aa2e-34c112bf6f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.798427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0195716-5f93-454d-9878-3287715614c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:49:54.798734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0195716-5f93-454d-9878-3287715614c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.798828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55e2f7-5c3e-40b5-aa2e-34c112bf6f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.799099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55e2f7-5c3e-40b5-aa2e-34c112bf6f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.799561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55e2f7-5c3e-40b5-aa2e-34c112bf6f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.800284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55e2f7-5c3e-40b5-aa2e-34c112bf6f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.801322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0195716-5f93-454d-9878-3287715614c0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.803102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55e2f7-5c3e-40b5-aa2e-34c112bf6f75 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.816350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 792/1577] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:49:54.817013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0195716-5f93-454d-9878-3287715614c0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.830459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0195716-5f93-454d-9878-3287715614c0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.837852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180556c2-d3a7-48bf-9049-ccb60697e062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:54.839434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180556c2-d3a7-48bf-9049-ccb60697e062 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:54.839434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180556c2-d3a7-48bf-9049-ccb60697e062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:54.839760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180556c2-d3a7-48bf-9049-ccb60697e062 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:54.840568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180556c2-d3a7-48bf-9049-ccb60697e062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:54.842505 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180556c2-d3a7-48bf-9049-ccb60697e062 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:54.886680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 786/1578] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/projects => generated 403 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:54.887537 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55e2f7-5c3e-40b5-aa2e-34c112bf6f75 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.943501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 793/1579] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:54.944345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180556c2-d3a7-48bf-9049-ccb60697e062 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:54.955675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55e2f7-5c3e-40b5-aa2e-34c112bf6f75 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:54.994430 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:55.021125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:55.028025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180556c2-d3a7-48bf-9049-ccb60697e062 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.033715 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74841480-036d-40df-abe7-6d9926f7102e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.037529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9dbaa-5c98-49b0-b293-81b793cf84c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.037672 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74841480-036d-40df-abe7-6d9926f7102e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.037672 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74841480-036d-40df-abe7-6d9926f7102e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.037672 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74841480-036d-40df-abe7-6d9926f7102e admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.037965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9dbaa-5c98-49b0-b293-81b793cf84c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.037965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9dbaa-5c98-49b0-b293-81b793cf84c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.037965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9dbaa-5c98-49b0-b293-81b793cf84c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.038223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74841480-036d-40df-abe7-6d9926f7102e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.038312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9dbaa-5c98-49b0-b293-81b793cf84c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.038406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74841480-036d-40df-abe7-6d9926f7102e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.038563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9dbaa-5c98-49b0-b293-81b793cf84c9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.154381 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 794/1580] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/users => generated 288 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:55.156456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138396b3-7ecb-4eba-94a8-9fab855b39a8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.156857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138396b3-7ecb-4eba-94a8-9fab855b39a8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.157153 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138396b3-7ecb-4eba-94a8-9fab855b39a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.188339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 787/1581] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:54 2022] POST /identity/v3/projects => generated 363 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:55.189500 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74841480-036d-40df-abe7-6d9926f7102e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.203356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-138396b3-7ecb-4eba-94a8-9fab855b39a8 None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:55.205315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-138396b3-7ecb-4eba-94a8-9fab855b39a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:55.268137 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74841480-036d-40df-abe7-6d9926f7102e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.273018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf1a7f-0de2-4763-801b-ed084977bda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.273697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf1a7f-0de2-4763-801b-ed084977bda2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.273984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf1a7f-0de2-4763-801b-ed084977bda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.274266 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf1a7f-0de2-4763-801b-ed084977bda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.275005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf1a7f-0de2-4763-801b-ed084977bda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.276689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf1a7f-0de2-4763-801b-ed084977bda2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.279031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 795/1582] 10.222.0.88 () {58 vars in 1059 bytes} [Mon Jul 18 16:49:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:55.279656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138396b3-7ecb-4eba-94a8-9fab855b39a8 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.290444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138396b3-7ecb-4eba-94a8-9fab855b39a8 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.293477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 788/1583] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:55.294327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf1a7f-0de2-4763-801b-ed084977bda2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.298353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2ccde-1292-4b9d-aeaa-6853c16e7e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.299384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2ccde-1292-4b9d-aeaa-6853c16e7e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.299689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2ccde-1292-4b9d-aeaa-6853c16e7e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.299947 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2ccde-1292-4b9d-aeaa-6853c16e7e95 admin admin] PATH_INFO: `/v3/projects/6299ea3e56e84d3cbd2e8f74ef62510e/users/f2d1f86034b94e289bb1fc754f02f4ac/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.306839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf1a7f-0de2-4763-801b-ed084977bda2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.318480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2ccde-1292-4b9d-aeaa-6853c16e7e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6299ea3e56e84d3cbd2e8f74ef62510e, user_id=f2d1f86034b94e289bb1fc754f02f4ac, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.319549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-9bf2ccde-1292-4b9d-aeaa-6853c16e7e95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:49:55.319549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 18 16:49:55.319549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: Jul 18 16:49:55.319808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17867ffa-5f36-431f-a7bd-e49b3fc2e21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.320346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17867ffa-5f36-431f-a7bd-e49b3fc2e21b admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.320749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17867ffa-5f36-431f-a7bd-e49b3fc2e21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.321099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17867ffa-5f36-431f-a7bd-e49b3fc2e21b admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.321526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2ccde-1292-4b9d-aeaa-6853c16e7e95 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.321837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17867ffa-5f36-431f-a7bd-e49b3fc2e21b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.323774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17867ffa-5f36-431f-a7bd-e49b3fc2e21b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.452219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 796/1584] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:55 2022] PUT /identity/v3/projects/6299ea3e56e84d3cbd2e8f74ef62510e/users/f2d1f86034b94e289bb1fc754f02f4ac/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:55.453083 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2ccde-1292-4b9d-aeaa-6853c16e7e95 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.478366 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 789/1585] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:55 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:55.479191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17867ffa-5f36-431f-a7bd-e49b3fc2e21b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.523782 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2ccde-1292-4b9d-aeaa-6853c16e7e95 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.543283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:55.552818 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17867ffa-5f36-431f-a7bd-e49b3fc2e21b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.557691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9c100-daf2-42e2-9510-d4370ec897ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.558497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9c100-daf2-42e2-9510-d4370ec897ed admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.559002 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9c100-daf2-42e2-9510-d4370ec897ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.559130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9c100-daf2-42e2-9510-d4370ec897ed admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.559751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9c100-daf2-42e2-9510-d4370ec897ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.560782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-3bb9c100-daf2-42e2-9510-d4370ec897ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:49:55.560782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 18 16:49:55.560782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: Jul 18 16:49:55.562027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9c100-daf2-42e2-9510-d4370ec897ed admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.594282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3716a-28e7-4cc7-9b7d-d4868f902996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.595303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3716a-28e7-4cc7-9b7d-d4868f902996 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.595589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3716a-28e7-4cc7-9b7d-d4868f902996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.595867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3716a-28e7-4cc7-9b7d-d4868f902996 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.596574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3716a-28e7-4cc7-9b7d-d4868f902996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.598552 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3716a-28e7-4cc7-9b7d-d4868f902996 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.667768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 790/1586] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:55 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:55.667768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9c100-daf2-42e2-9510-d4370ec897ed admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.675578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9c100-daf2-42e2-9510-d4370ec897ed admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.680318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf03a5-42bb-43b6-86dd-055ff781145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.681023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf03a5-42bb-43b6-86dd-055ff781145c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.681315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf03a5-42bb-43b6-86dd-055ff781145c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.681614 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf03a5-42bb-43b6-86dd-055ff781145c admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.682355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf03a5-42bb-43b6-86dd-055ff781145c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.683951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf03a5-42bb-43b6-86dd-055ff781145c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.694542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 791/1587] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:55.695510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf03a5-42bb-43b6-86dd-055ff781145c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.698375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 797/1588] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:55 2022] POST /identity/v3/users => generated 302 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:55.699435 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3716a-28e7-4cc7-9b7d-d4868f902996 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.706394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf03a5-42bb-43b6-86dd-055ff781145c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.711769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de17e0-efb2-4a0b-b316-1ad2063bdd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.712440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de17e0-efb2-4a0b-b316-1ad2063bdd13 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.712721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de17e0-efb2-4a0b-b316-1ad2063bdd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.713097 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de17e0-efb2-4a0b-b316-1ad2063bdd13 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.713717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de17e0-efb2-4a0b-b316-1ad2063bdd13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.715885 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de17e0-efb2-4a0b-b316-1ad2063bdd13 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.778829 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3716a-28e7-4cc7-9b7d-d4868f902996 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.784616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00de8db-4757-42bc-afd2-d0f43935afdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.785308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00de8db-4757-42bc-afd2-d0f43935afdf service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.785601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00de8db-4757-42bc-afd2-d0f43935afdf service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.785881 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00de8db-4757-42bc-afd2-d0f43935afdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.800534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 792/1589] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:55 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:55.801497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de17e0-efb2-4a0b-b316-1ad2063bdd13 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.812637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de17e0-efb2-4a0b-b316-1ad2063bdd13 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.817529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789caaf-0c2a-48c5-b0fc-06dd8864e0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.818411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789caaf-0c2a-48c5-b0fc-06dd8864e0cc admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.818704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789caaf-0c2a-48c5-b0fc-06dd8864e0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.818985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789caaf-0c2a-48c5-b0fc-06dd8864e0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.819721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789caaf-0c2a-48c5-b0fc-06dd8864e0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.821514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789caaf-0c2a-48c5-b0fc-06dd8864e0cc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.833872 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 793/1590] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:55.835179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f8971-5a89-4438-ad7c-ba4b227aebe1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.835490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f8971-5a89-4438-ad7c-ba4b227aebe1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.835754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f8971-5a89-4438-ad7c-ba4b227aebe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.841842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:55.854579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7d8f8971-5a89-4438-ad7c-ba4b227aebe1 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:55.855890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7d8f8971-5a89-4438-ad7c-ba4b227aebe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:55.863765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 794/1591] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:55.864687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f8971-5a89-4438-ad7c-ba4b227aebe1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.869693 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00de8db-4757-42bc-afd2-d0f43935afdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.871393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00de8db-4757-42bc-afd2-d0f43935afdf service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.874879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f8971-5a89-4438-ad7c-ba4b227aebe1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.879636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8416a-f67a-4fae-b3f6-65a4c8cce6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.881481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8416a-f67a-4fae-b3f6-65a4c8cce6de admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.881890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 798/1592] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:55.882013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8416a-f67a-4fae-b3f6-65a4c8cce6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.882213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8416a-f67a-4fae-b3f6-65a4c8cce6de admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.882583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00de8db-4757-42bc-afd2-d0f43935afdf service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.882914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8416a-f67a-4fae-b3f6-65a4c8cce6de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.884866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8416a-f67a-4fae-b3f6-65a4c8cce6de admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.893189 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00de8db-4757-42bc-afd2-d0f43935afdf service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.899384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0cbcc-2b6c-4854-98a2-7639ad8eee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.899384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0cbcc-2b6c-4854-98a2-7639ad8eee91 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.899634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0cbcc-2b6c-4854-98a2-7639ad8eee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.899812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0cbcc-2b6c-4854-98a2-7639ad8eee91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.900553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0cbcc-2b6c-4854-98a2-7639ad8eee91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.902191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0cbcc-2b6c-4854-98a2-7639ad8eee91 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.913065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 799/1593] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:55.913986 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0cbcc-2b6c-4854-98a2-7639ad8eee91 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.928152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0cbcc-2b6c-4854-98a2-7639ad8eee91 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.930346 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d3b13-75cc-4c7a-9414-39fd3f192e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.931020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d3b13-75cc-4c7a-9414-39fd3f192e8f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.931353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d3b13-75cc-4c7a-9414-39fd3f192e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.931572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d3b13-75cc-4c7a-9414-39fd3f192e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.932303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d3b13-75cc-4c7a-9414-39fd3f192e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.934096 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d3b13-75cc-4c7a-9414-39fd3f192e8f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.946163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 800/1594] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:55.948076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d3b13-75cc-4c7a-9414-39fd3f192e8f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.961266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d3b13-75cc-4c7a-9414-39fd3f192e8f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:55.966277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf4b45-370c-4785-a6a7-f7655450e8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:55.966938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf4b45-370c-4785-a6a7-f7655450e8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:55.967218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf4b45-370c-4785-a6a7-f7655450e8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:55.967494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf4b45-370c-4785-a6a7-f7655450e8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:55.968232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf4b45-370c-4785-a6a7-f7655450e8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:55.969853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf4b45-370c-4785-a6a7-f7655450e8b3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:55.980776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 801/1595] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:55.981589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf4b45-370c-4785-a6a7-f7655450e8b3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.989143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 795/1596] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:55 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:55.990459 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8416a-f67a-4fae-b3f6-65a4c8cce6de admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:55.994254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf4b45-370c-4785-a6a7-f7655450e8b3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.003919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77946f-4f87-4ae8-b724-b6d244ce0572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.004407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8416a-f67a-4fae-b3f6-65a4c8cce6de admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.004620 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77946f-4f87-4ae8-b724-b6d244ce0572 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.004934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77946f-4f87-4ae8-b724-b6d244ce0572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.005779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77946f-4f87-4ae8-b724-b6d244ce0572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.006674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77946f-4f87-4ae8-b724-b6d244ce0572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.008313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77946f-4f87-4ae8-b724-b6d244ce0572 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.010005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb035d-af05-4a7d-a7dc-6771fc7e9265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.011124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb035d-af05-4a7d-a7dc-6771fc7e9265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.011261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb035d-af05-4a7d-a7dc-6771fc7e9265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.011508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb035d-af05-4a7d-a7dc-6771fc7e9265 admin admin] PATH_INFO: `/v3/projects/7a7f46101a294535bfc2a13a23e7a884/users/d528ebf2e8b64a51bc92b9f636902da2/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.021474 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 802/1597] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:56.023098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77946f-4f87-4ae8-b724-b6d244ce0572 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.027991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb035d-af05-4a7d-a7dc-6771fc7e9265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7f46101a294535bfc2a13a23e7a884, user_id=d528ebf2e8b64a51bc92b9f636902da2, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.029386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:56.030701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfb035d-af05-4a7d-a7dc-6771fc7e9265 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.082282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77946f-4f87-4ae8-b724-b6d244ce0572 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.086801 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a9291-936d-4411-a154-3b82cb08e68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.087469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a9291-936d-4411-a154-3b82cb08e68c service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.087748 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a9291-936d-4411-a154-3b82cb08e68c service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.088024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a9291-936d-4411-a154-3b82cb08e68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.099293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a9291-936d-4411-a154-3b82cb08e68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.101575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a9291-936d-4411-a154-3b82cb08e68c service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.109317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 796/1598] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:55 2022] PUT /identity/v3/projects/7a7f46101a294535bfc2a13a23e7a884/users/d528ebf2e8b64a51bc92b9f636902da2/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:56.110312 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb035d-af05-4a7d-a7dc-6771fc7e9265 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.115517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 803/1599] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:56.119098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a9291-936d-4411-a154-3b82cb08e68c service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.137130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a9291-936d-4411-a154-3b82cb08e68c service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.200137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110c0c3-eca8-4b21-a2a6-88cd3f502619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.201008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110c0c3-eca8-4b21-a2a6-88cd3f502619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.201380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110c0c3-eca8-4b21-a2a6-88cd3f502619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.201740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110c0c3-eca8-4b21-a2a6-88cd3f502619 admin admin] PATH_INFO: `/v3/projects/e8bca7eae3a348069131e97b048412b5/users/6890f2e62cfd46999791d4a10a81f95c/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.203802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb035d-af05-4a7d-a7dc-6771fc7e9265 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.210471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efca55-bde2-49bb-bc8e-2965cdb43040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.211537 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efca55-bde2-49bb-bc8e-2965cdb43040 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.211983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efca55-bde2-49bb-bc8e-2965cdb43040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.212405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efca55-bde2-49bb-bc8e-2965cdb43040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.213318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efca55-bde2-49bb-bc8e-2965cdb43040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.216953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efca55-bde2-49bb-bc8e-2965cdb43040 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.221948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110c0c3-eca8-4b21-a2a6-88cd3f502619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8bca7eae3a348069131e97b048412b5, user_id=6890f2e62cfd46999791d4a10a81f95c, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.226535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110c0c3-eca8-4b21-a2a6-88cd3f502619 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.229468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 797/1600] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:56.234043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efca55-bde2-49bb-bc8e-2965cdb43040 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.242625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efca55-bde2-49bb-bc8e-2965cdb43040 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.248278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e131568-bcd3-4b6d-b66e-ffbb043ce516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.249059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e131568-bcd3-4b6d-b66e-ffbb043ce516 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.249345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e131568-bcd3-4b6d-b66e-ffbb043ce516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.249699 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e131568-bcd3-4b6d-b66e-ffbb043ce516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.250545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e131568-bcd3-4b6d-b66e-ffbb043ce516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.252069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e131568-bcd3-4b6d-b66e-ffbb043ce516 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.263755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 798/1601] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:56.264597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e131568-bcd3-4b6d-b66e-ffbb043ce516 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.277908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e131568-bcd3-4b6d-b66e-ffbb043ce516 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.282819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b1e50-9a36-4177-a454-4f758d7e8937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.283533 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b1e50-9a36-4177-a454-4f758d7e8937 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.285840 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b1e50-9a36-4177-a454-4f758d7e8937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.285840 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b1e50-9a36-4177-a454-4f758d7e8937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.285840 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b1e50-9a36-4177-a454-4f758d7e8937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.286493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b1e50-9a36-4177-a454-4f758d7e8937 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.298655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 799/1602] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:56.299720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b1e50-9a36-4177-a454-4f758d7e8937 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.310662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b1e50-9a36-4177-a454-4f758d7e8937 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.320375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 804/1603] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:56 2022] PUT /identity/v3/projects/e8bca7eae3a348069131e97b048412b5/users/6890f2e62cfd46999791d4a10a81f95c/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:56.322196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf1634-7f98-446a-8a7a-a444f8118e67 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.322936 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf1634-7f98-446a-8a7a-a444f8118e67 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.323324 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf1634-7f98-446a-8a7a-a444f8118e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.357106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7faf1634-7f98-446a-8a7a-a444f8118e67 None None] MFA Rules not processed for user `d528ebf2e8b64a51bc92b9f636902da2`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:56.358459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7faf1634-7f98-446a-8a7a-a444f8118e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:56.384836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde960b-0db6-4ddd-a39c-1dd7dcaff33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.384836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde960b-0db6-4ddd-a39c-1dd7dcaff33d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.384836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde960b-0db6-4ddd-a39c-1dd7dcaff33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.384836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde960b-0db6-4ddd-a39c-1dd7dcaff33d admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.385724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde960b-0db6-4ddd-a39c-1dd7dcaff33d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.385724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde960b-0db6-4ddd-a39c-1dd7dcaff33d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.396766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 800/1604] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:56.397677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde960b-0db6-4ddd-a39c-1dd7dcaff33d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.410535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde960b-0db6-4ddd-a39c-1dd7dcaff33d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.417569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98fcc5-5e28-42d0-8e8a-a76314ec1938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.418673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98fcc5-5e28-42d0-8e8a-a76314ec1938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.418964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98fcc5-5e28-42d0-8e8a-a76314ec1938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.419404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98fcc5-5e28-42d0-8e8a-a76314ec1938 admin admin] PATH_INFO: `/v3/projects/b61afdae54fc4f76b7dd31798c4226f4/users/baaf254c2b28444bab7f1c7cc3fbc42c/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.440424 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98fcc5-5e28-42d0-8e8a-a76314ec1938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b61afdae54fc4f76b7dd31798c4226f4, user_id=baaf254c2b28444bab7f1c7cc3fbc42c, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.444368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98fcc5-5e28-42d0-8e8a-a76314ec1938 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.455185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 805/1605] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:56.455185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf1634-7f98-446a-8a7a-a444f8118e67 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.466228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf1634-7f98-446a-8a7a-a444f8118e67 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.475548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f56ef-fa40-4505-b02d-e14c7167c0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.477001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f56ef-fa40-4505-b02d-e14c7167c0fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.477301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f56ef-fa40-4505-b02d-e14c7167c0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.477582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f56ef-fa40-4505-b02d-e14c7167c0fc admin admin] PATH_INFO: `/v3/projects/70335d12ffff4e399dd18661949c3b97/users/948d7854fc784c4689b4c29ddcb13224/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.496171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f56ef-fa40-4505-b02d-e14c7167c0fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=70335d12ffff4e399dd18661949c3b97, user_id=948d7854fc784c4689b4c29ddcb13224, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.501112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f56ef-fa40-4505-b02d-e14c7167c0fc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.507681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:56.510918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:56.515991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 801/1606] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:56 2022] PUT /identity/v3/projects/b61afdae54fc4f76b7dd31798c4226f4/users/baaf254c2b28444bab7f1c7cc3fbc42c/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:56.519181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98fcc5-5e28-42d0-8e8a-a76314ec1938 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.577951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98fcc5-5e28-42d0-8e8a-a76314ec1938 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.582762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bbc88-3a95-405c-b97d-061c52d43b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.583484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bbc88-3a95-405c-b97d-061c52d43b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.583782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bbc88-3a95-405c-b97d-061c52d43b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.584040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bbc88-3a95-405c-b97d-061c52d43b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.584774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bbc88-3a95-405c-b97d-061c52d43b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.586407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bbc88-3a95-405c-b97d-061c52d43b96 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.597194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 802/1607] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:56.598305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5473bf-44a1-4683-a170-b72c2ff7b522 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.598659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5473bf-44a1-4683-a170-b72c2ff7b522 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.599033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5473bf-44a1-4683-a170-b72c2ff7b522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.608256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 806/1608] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:56 2022] PUT /identity/v3/projects/70335d12ffff4e399dd18661949c3b97/users/948d7854fc784c4689b4c29ddcb13224/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:56.620764 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f56ef-fa40-4505-b02d-e14c7167c0fc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.620959 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2b5473bf-44a1-4683-a170-b72c2ff7b522 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:56.620959 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2b5473bf-44a1-4683-a170-b72c2ff7b522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:56.681273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f56ef-fa40-4505-b02d-e14c7167c0fc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.686514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf1854-f67b-400d-87bb-2aec847df726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.687408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf1854-f67b-400d-87bb-2aec847df726 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.687923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf1854-f67b-400d-87bb-2aec847df726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.688213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf1854-f67b-400d-87bb-2aec847df726 admin admin] PATH_INFO: `/v3/projects/44f98c5e4b8645048b138492c4d21992/users/330ff9790b3c4972b98521f1f6a52358/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.693760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 803/1609] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:56.695141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141bf94-86c6-4c07-afeb-cf3f8324683b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.696922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141bf94-86c6-4c07-afeb-cf3f8324683b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.696922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141bf94-86c6-4c07-afeb-cf3f8324683b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.697613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 804/1610] 10.222.0.88 () {54 vars in 962 bytes} [Mon Jul 18 16:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:49:56.698229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141bf94-86c6-4c07-afeb-cf3f8324683b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.711601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf1854-f67b-400d-87bb-2aec847df726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f98c5e4b8645048b138492c4d21992, user_id=330ff9790b3c4972b98521f1f6a52358, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.713987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf1854-f67b-400d-87bb-2aec847df726 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.777718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141bf94-86c6-4c07-afeb-cf3f8324683b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.781749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecab36-36c0-4d5e-bdab-a2bf38a33db5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.782720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecab36-36c0-4d5e-bdab-a2bf38a33db5 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.783099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecab36-36c0-4d5e-bdab-a2bf38a33db5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.783321 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ecab36-36c0-4d5e-bdab-a2bf38a33db5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.797815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 807/1611] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:56 2022] PUT /identity/v3/projects/44f98c5e4b8645048b138492c4d21992/users/330ff9790b3c4972b98521f1f6a52358/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:56.799424 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fca49-2355-4296-9859-3240c05cff16 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.799801 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fca49-2355-4296-9859-3240c05cff16 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.800078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fca49-2355-4296-9859-3240c05cff16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.826685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-122fca49-2355-4296-9859-3240c05cff16 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:56.827928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-122fca49-2355-4296-9859-3240c05cff16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:56.883268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecab36-36c0-4d5e-bdab-a2bf38a33db5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:56.885258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ecab36-36c0-4d5e-bdab-a2bf38a33db5 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:56.896400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 805/1612] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:56.898068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecab36-36c0-4d5e-bdab-a2bf38a33db5 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:56.902758 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 808/1613] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:56.904568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c3893-f595-4ab2-a642-62a03556b358 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.905116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c3893-f595-4ab2-a642-62a03556b358 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.905586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c3893-f595-4ab2-a642-62a03556b358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.921810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ecab36-36c0-4d5e-bdab-a2bf38a33db5 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:56.924822 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:56.944586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-945c3893-f595-4ab2-a642-62a03556b358 None None] MFA Rules not processed for user `948d7854fc784c4689b4c29ddcb13224`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:56.945987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-945c3893-f595-4ab2-a642-62a03556b358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:56.954871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cb4ce-bb81-488a-b8ac-3a1d8ffd11c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:56.956361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cb4ce-bb81-488a-b8ac-3a1d8ffd11c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:56.956836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cb4ce-bb81-488a-b8ac-3a1d8ffd11c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:56.957704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285cb4ce-bb81-488a-b8ac-3a1d8ffd11c7 admin admin] PATH_INFO: `/v3/projects/c956cdbbf8d14dfe89b00a63a98dcc2e/users/f169cd11553546adb4aceb8c0672a084/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:56.993496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cb4ce-bb81-488a-b8ac-3a1d8ffd11c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c956cdbbf8d14dfe89b00a63a98dcc2e, user_id=f169cd11553546adb4aceb8c0672a084, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:57.000409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285cb4ce-bb81-488a-b8ac-3a1d8ffd11c7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:57.030670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 809/1614] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:57.034418 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094f38d-3f21-4995-b0b9-6eb663e38449 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:57.035042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094f38d-3f21-4995-b0b9-6eb663e38449 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:57.035321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094f38d-3f21-4995-b0b9-6eb663e38449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:57.052162 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:57.058788 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1094f38d-3f21-4995-b0b9-6eb663e38449 None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:57.060132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1094f38d-3f21-4995-b0b9-6eb663e38449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:57.087634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 806/1615] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:56 2022] PUT /identity/v3/projects/c956cdbbf8d14dfe89b00a63a98dcc2e/users/f169cd11553546adb4aceb8c0672a084/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:57.101027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cb4ce-bb81-488a-b8ac-3a1d8ffd11c7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:57.102804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-285cb4ce-bb81-488a-b8ac-3a1d8ffd11c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:57.178452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 810/1616] 10.222.0.88 () {58 vars in 1059 bytes} [Mon Jul 18 16:49:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:57.179739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537b9a-f1b1-478a-9b97-f1fa4db739f1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:57.180030 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537b9a-f1b1-478a-9b97-f1fa4db739f1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:57.180307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537b9a-f1b1-478a-9b97-f1fa4db739f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:57.194367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285cb4ce-bb81-488a-b8ac-3a1d8ffd11c7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:57.198943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed331b26-047e-4314-9b19-eba4f72ec10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:57.200443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed331b26-047e-4314-9b19-eba4f72ec10f service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:57.200443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed331b26-047e-4314-9b19-eba4f72ec10f service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:57.200443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed331b26-047e-4314-9b19-eba4f72ec10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:57.223215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-84537b9a-f1b1-478a-9b97-f1fa4db739f1 None None] MFA Rules not processed for user `330ff9790b3c4972b98521f1f6a52358`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:57.224954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-84537b9a-f1b1-478a-9b97-f1fa4db739f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:57.313975 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed331b26-047e-4314-9b19-eba4f72ec10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:57.315650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed331b26-047e-4314-9b19-eba4f72ec10f service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:57.329868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 811/1617] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:57 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:57.330138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 807/1618] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:57.336362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed331b26-047e-4314-9b19-eba4f72ec10f service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:57.336492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84537b9a-f1b1-478a-9b97-f1fa4db739f1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:57.348343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed331b26-047e-4314-9b19-eba4f72ec10f service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:57.350990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84537b9a-f1b1-478a-9b97-f1fa4db739f1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:57.355012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fe6bd-fed4-436c-ad09-a662255c28d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:57.355836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe6bd-fed4-436c-ad09-a662255c28d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:57.356230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe6bd-fed4-436c-ad09-a662255c28d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:57.356774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fe6bd-fed4-436c-ad09-a662255c28d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:57.357453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee492a-29fd-4498-a3b0-a514b0784097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:57.357753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fe6bd-fed4-436c-ad09-a662255c28d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:57.358769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee492a-29fd-4498-a3b0-a514b0784097 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:57.359026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee492a-29fd-4498-a3b0-a514b0784097 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:57.359687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee492a-29fd-4498-a3b0-a514b0784097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:57.360523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fe6bd-fed4-436c-ad09-a662255c28d4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:57.377594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee492a-29fd-4498-a3b0-a514b0784097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:57.379289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee492a-29fd-4498-a3b0-a514b0784097 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:57.390290 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 812/1619] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:57.391589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7abf27-8b19-421c-8224-971772b66190 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:57.391876 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7abf27-8b19-421c-8224-971772b66190 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:57.392159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7abf27-8b19-421c-8224-971772b66190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:57.423118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8c7abf27-8b19-421c-8224-971772b66190 None None] MFA Rules not processed for user `f169cd11553546adb4aceb8c0672a084`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:57.424402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8c7abf27-8b19-421c-8224-971772b66190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:57.462318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 808/1620] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:57 2022] POST /identity/v3/projects => generated 403 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:57.462318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60775558-baab-41c0-be86-ff6681ba98ef None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:57.462318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60775558-baab-41c0-be86-ff6681ba98ef None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:57.462318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60775558-baab-41c0-be86-ff6681ba98ef None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:57.463407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 809/1621] 10.222.0.88 () {54 vars in 912 bytes} [Mon Jul 18 16:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:49:57.464969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60775558-baab-41c0-be86-ff6681ba98ef None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:57.525052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 813/1622] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:57.525052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7abf27-8b19-421c-8224-971772b66190 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:57.532390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:57.537794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7abf27-8b19-421c-8224-971772b66190 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:57.556560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:57.562690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60775558-baab-41c0-be86-ff6681ba98ef None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:57.567566 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a8c2b-2034-4405-b7e1-30ab65b478fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:57.568779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a8c2b-2034-4405-b7e1-30ab65b478fe admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:57.568779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a8c2b-2034-4405-b7e1-30ab65b478fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:57.568779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a8c2b-2034-4405-b7e1-30ab65b478fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:57.569820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a8c2b-2034-4405-b7e1-30ab65b478fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:57.572273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a8c2b-2034-4405-b7e1-30ab65b478fe admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:57.649528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c31eb2-2a68-41a4-897f-481e7feeb938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:57.650534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c31eb2-2a68-41a4-897f-481e7feeb938 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:57.651136 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c31eb2-2a68-41a4-897f-481e7feeb938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:57.651420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c31eb2-2a68-41a4-897f-481e7feeb938 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:57.652129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c31eb2-2a68-41a4-897f-481e7feeb938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:57.654564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c31eb2-2a68-41a4-897f-481e7feeb938 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:57.742226 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 810/1623] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:49:57 2022] POST /identity/v3/projects => generated 365 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:57.744378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a55a5-cc2c-49be-9b4d-16f315f4e025 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:57.744687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a55a5-cc2c-49be-9b4d-16f315f4e025 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:57.745051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a55a5-cc2c-49be-9b4d-16f315f4e025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:57.766290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e96a55a5-cc2c-49be-9b4d-16f315f4e025 None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:57.767527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e96a55a5-cc2c-49be-9b4d-16f315f4e025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:57.795669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 814/1624] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:57 2022] POST /identity/v3/users => generated 308 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:57.796617 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c31eb2-2a68-41a4-897f-481e7feeb938 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:57.859135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 811/1625] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:49:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:57.860109 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a55a5-cc2c-49be-9b4d-16f315f4e025 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:57.948016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a55a5-cc2c-49be-9b4d-16f315f4e025 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:57.956801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec1ba7-8c5f-4cde-8488-cee498ce3475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:57.959158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec1ba7-8c5f-4cde-8488-cee498ce3475 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:57.959522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec1ba7-8c5f-4cde-8488-cee498ce3475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:57.959830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec1ba7-8c5f-4cde-8488-cee498ce3475 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:57.960497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec1ba7-8c5f-4cde-8488-cee498ce3475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:57.963074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec1ba7-8c5f-4cde-8488-cee498ce3475 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:58.079869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 812/1626] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:49:57 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:49:58.081059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec1ba7-8c5f-4cde-8488-cee498ce3475 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:58.082330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c31eb2-2a68-41a4-897f-481e7feeb938 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:58.093286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cef463-8fab-4c08-9e66-b9c277bacae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:58.095331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec1ba7-8c5f-4cde-8488-cee498ce3475 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:58.095873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cef463-8fab-4c08-9e66-b9c277bacae1 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:58.096284 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cef463-8fab-4c08-9e66-b9c277bacae1 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:58.096659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cef463-8fab-4c08-9e66-b9c277bacae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:58.102689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79b2c4-811c-43a9-a3b0-ac8f7915ecb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:58.103278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79b2c4-811c-43a9-a3b0-ac8f7915ecb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:58.103857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79b2c4-811c-43a9-a3b0-ac8f7915ecb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:58.105408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79b2c4-811c-43a9-a3b0-ac8f7915ecb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:58.105567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79b2c4-811c-43a9-a3b0-ac8f7915ecb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:58.108923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79b2c4-811c-43a9-a3b0-ac8f7915ecb1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:58.123551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 813/1627] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:58.125106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79b2c4-811c-43a9-a3b0-ac8f7915ecb1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:58.181955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cef463-8fab-4c08-9e66-b9c277bacae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:58.183960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cef463-8fab-4c08-9e66-b9c277bacae1 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:58.196452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 815/1628] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:58.197704 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cef463-8fab-4c08-9e66-b9c277bacae1 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:58.209803 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79b2c4-811c-43a9-a3b0-ac8f7915ecb1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:58.210726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cef463-8fab-4c08-9e66-b9c277bacae1 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:58.214819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63a065-516e-4967-8eca-80b539738d7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:58.215491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63a065-516e-4967-8eca-80b539738d7f service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:58.215782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63a065-516e-4967-8eca-80b539738d7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:58.216158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63a065-516e-4967-8eca-80b539738d7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:58.218193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9006d-cb09-4324-9673-4c3293b4ce18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:58.219391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9006d-cb09-4324-9673-4c3293b4ce18 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:58.219954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9006d-cb09-4324-9673-4c3293b4ce18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:58.220655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9006d-cb09-4324-9673-4c3293b4ce18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:58.221505 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9006d-cb09-4324-9673-4c3293b4ce18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:58.223438 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9006d-cb09-4324-9673-4c3293b4ce18 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:58.225903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:58.233976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63a065-516e-4967-8eca-80b539738d7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:58.237701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63a065-516e-4967-8eca-80b539738d7f service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:58.239773 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 816/1629] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:58.241225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9006d-cb09-4324-9673-4c3293b4ce18 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:58.257622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 814/1630] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:58.258328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63a065-516e-4967-8eca-80b539738d7f service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:58.264121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9006d-cb09-4324-9673-4c3293b4ce18 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:58.272478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75223558-074a-4587-aaa2-14438eebecac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:58.274175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75223558-074a-4587-aaa2-14438eebecac admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:58.274641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75223558-074a-4587-aaa2-14438eebecac admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:58.275423 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75223558-074a-4587-aaa2-14438eebecac admin admin] PATH_INFO: `/v3/projects/14f3713ecf7246a9a286e57577014bdc/users/9693a07b07bf43d6b59090150df8b377/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:58.279749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63a065-516e-4967-8eca-80b539738d7f service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:58.297026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180852d-d1d4-4d3e-8109-a4a3f232c147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:58.300185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180852d-d1d4-4d3e-8109-a4a3f232c147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:58.300185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180852d-d1d4-4d3e-8109-a4a3f232c147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:58.300185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180852d-d1d4-4d3e-8109-a4a3f232c147 admin admin] PATH_INFO: `/v3/projects/d0ad5e03cebc44d6b2df0d30505083b4/users/b95c1b4fcf1f4daa8837757d5169d36a/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:58.321031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75223558-074a-4587-aaa2-14438eebecac admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f3713ecf7246a9a286e57577014bdc, user_id=9693a07b07bf43d6b59090150df8b377, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:58.333145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75223558-074a-4587-aaa2-14438eebecac admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:58.334504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180852d-d1d4-4d3e-8109-a4a3f232c147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ad5e03cebc44d6b2df0d30505083b4, user_id=b95c1b4fcf1f4daa8837757d5169d36a, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:58.343556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2180852d-d1d4-4d3e-8109-a4a3f232c147 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:58.347332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:58.459650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 817/1631] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:58 2022] PUT /identity/v3/projects/14f3713ecf7246a9a286e57577014bdc/users/9693a07b07bf43d6b59090150df8b377/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:58.459650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901894a2-8254-406b-a4cd-f6d3532b1b65 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:58.459650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901894a2-8254-406b-a4cd-f6d3532b1b65 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:58.459650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901894a2-8254-406b-a4cd-f6d3532b1b65 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:58.462066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 818/1632] 10.222.0.88 () {54 vars in 912 bytes} [Mon Jul 18 16:49:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:49:58.464932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901894a2-8254-406b-a4cd-f6d3532b1b65 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:58.526448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 815/1633] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:58 2022] PUT /identity/v3/projects/d0ad5e03cebc44d6b2df0d30505083b4/users/b95c1b4fcf1f4daa8837757d5169d36a/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:58.528587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd4f9e-a1e0-4cb6-b7df-4454b0385745 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:58.528947 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd4f9e-a1e0-4cb6-b7df-4454b0385745 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:58.534464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd4f9e-a1e0-4cb6-b7df-4454b0385745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:58.602476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901894a2-8254-406b-a4cd-f6d3532b1b65 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:58.607091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-56bd4f9e-a1e0-4cb6-b7df-4454b0385745 None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:58.615252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-56bd4f9e-a1e0-4cb6-b7df-4454b0385745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:58.677822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9858a-dea6-4ea9-8757-2bd91a9fd3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:58.678565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9858a-dea6-4ea9-8757-2bd91a9fd3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:58.678854 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9858a-dea6-4ea9-8757-2bd91a9fd3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:58.679132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9858a-dea6-4ea9-8757-2bd91a9fd3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:58.679934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9858a-dea6-4ea9-8757-2bd91a9fd3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:59.149137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9858a-dea6-4ea9-8757-2bd91a9fd3b8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:59.149137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 819/1634] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:59.149137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9858a-dea6-4ea9-8757-2bd91a9fd3b8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:59.149137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9858a-dea6-4ea9-8757-2bd91a9fd3b8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:59.152396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea80c4-f846-4882-9757-dd23c39ea473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:59.152396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea80c4-f846-4882-9757-dd23c39ea473 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:59.152396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea80c4-f846-4882-9757-dd23c39ea473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:59.152396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea80c4-f846-4882-9757-dd23c39ea473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:59.154715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea80c4-f846-4882-9757-dd23c39ea473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:59.154715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea80c4-f846-4882-9757-dd23c39ea473 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:59.154715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 820/1635] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:49:59.154715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea80c4-f846-4882-9757-dd23c39ea473 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:59.154715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea80c4-f846-4882-9757-dd23c39ea473 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:59.157372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:59.157929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 816/1636] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:49:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:59.157929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd4f9e-a1e0-4cb6-b7df-4454b0385745 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:59.158993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:59.158993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:59.158993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] PATH_INFO: `/v3/users/4f6db5c6b0b64503afc740f5c4748ad1` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:59.158993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f6db5c6b0b64503afc740f5c4748ad1)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:59.158993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: WARNING py.warnings [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:49:59.158993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: warnings.warn(msg) Jul 18 16:49:59.158993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: Jul 18 16:49:59.160366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:59.160532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd4f9e-a1e0-4cb6-b7df-4454b0385745 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:59.160532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2197e4-090d-4792-9596-9d32c109bc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:59.160532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2197e4-090d-4792-9596-9d32c109bc7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:59.160532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2197e4-090d-4792-9596-9d32c109bc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:59.162082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2197e4-090d-4792-9596-9d32c109bc7d admin admin] PATH_INFO: `/v3/projects/14f3713ecf7246a9a286e57577014bdc/users/9693a07b07bf43d6b59090150df8b377/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:59.162082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2197e4-090d-4792-9596-9d32c109bc7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f3713ecf7246a9a286e57577014bdc, user_id=9693a07b07bf43d6b59090150df8b377, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:59.162082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2197e4-090d-4792-9596-9d32c109bc7d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:59.162082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 817/1637] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:58 2022] PUT /identity/v3/projects/14f3713ecf7246a9a286e57577014bdc/users/9693a07b07bf43d6b59090150df8b377/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:59.162082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2197e4-090d-4792-9596-9d32c109bc7d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:59.162868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2197e4-090d-4792-9596-9d32c109bc7d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:59.162868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a12170-8f91-4a33-b878-c4007febda5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:59.162868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a12170-8f91-4a33-b878-c4007febda5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:59.162868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a12170-8f91-4a33-b878-c4007febda5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:59.163393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a12170-8f91-4a33-b878-c4007febda5d admin admin] PATH_INFO: `/v3/projects/d0ad5e03cebc44d6b2df0d30505083b4/users/b95c1b4fcf1f4daa8837757d5169d36a/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:59.163393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a12170-8f91-4a33-b878-c4007febda5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ad5e03cebc44d6b2df0d30505083b4, user_id=b95c1b4fcf1f4daa8837757d5169d36a, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:59.163393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a12170-8f91-4a33-b878-c4007febda5d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:59.163393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 818/1638] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:49:58 2022] PUT /identity/v3/projects/d0ad5e03cebc44d6b2df0d30505083b4/users/b95c1b4fcf1f4daa8837757d5169d36a/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:59.163393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a12170-8f91-4a33-b878-c4007febda5d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:59.163841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a12170-8f91-4a33-b878-c4007febda5d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:59.163841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4618e4c-f301-4491-b540-1437fe3844e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:59.163841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4618e4c-f301-4491-b540-1437fe3844e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:59.163841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4618e4c-f301-4491-b540-1437fe3844e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:59.165681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4618e4c-f301-4491-b540-1437fe3844e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:59.165681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:59.165681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4618e4c-f301-4491-b540-1437fe3844e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:59.165681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4618e4c-f301-4491-b540-1437fe3844e7 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:59.165681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 819/1639] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:59.165681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb746a14-c5dc-4593-9e7d-335592da93ec None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:59.169357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb746a14-c5dc-4593-9e7d-335592da93ec None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:59.169357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb746a14-c5dc-4593-9e7d-335592da93ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:59.169357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-fb746a14-c5dc-4593-9e7d-335592da93ec None None] MFA Rules not processed for user `9693a07b07bf43d6b59090150df8b377`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:59.169357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fb746a14-c5dc-4593-9e7d-335592da93ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:59.279888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f6db5c6b0b64503afc740f5c4748ad1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:59.305409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f6db5c6b0b64503afc740f5c4748ad1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:59.330765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 820/1640] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:59.332670 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0758a-b090-4f1e-a914-c7b6d25b431b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:59.333214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0758a-b090-4f1e-a914-c7b6d25b431b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:59.333648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0758a-b090-4f1e-a914-c7b6d25b431b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:59.375932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f6db5c6b0b64503afc740f5c4748ad1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:59.394755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6ab0758a-b090-4f1e-a914-c7b6d25b431b None None] MFA Rules not processed for user `b95c1b4fcf1f4daa8837757d5169d36a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:59.396760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6ab0758a-b090-4f1e-a914-c7b6d25b431b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:59.497281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 821/1641] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:49:59.497281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0758a-b090-4f1e-a914-c7b6d25b431b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:59.500349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f6db5c6b0b64503afc740f5c4748ad1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:59.502921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1b479e04-a308-4add-8fd8-e084cd12daa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f6db5c6b0b64503afc740f5c4748ad1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:49:59.513623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:59.522501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 821/1642] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:49:58 2022] DELETE /identity/v3/users/4f6db5c6b0b64503afc740f5c4748ad1 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:49:59.526958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7aa1-ce88-4e12-b62a-22372efeb127 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:59.527507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7aa1-ce88-4e12-b62a-22372efeb127 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:59.528030 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7aa1-ce88-4e12-b62a-22372efeb127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:59.529908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0758a-b090-4f1e-a914-c7b6d25b431b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:59.565678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-23bb7aa1-ce88-4e12-b62a-22372efeb127 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:49:59.567351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-23bb7aa1-ce88-4e12-b62a-22372efeb127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:59.666784 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831643bf-c741-4d22-938a-bd4abbd706f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:59.672493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831643bf-c741-4d22-938a-bd4abbd706f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:59.672925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831643bf-c741-4d22-938a-bd4abbd706f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:59.673219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831643bf-c741-4d22-938a-bd4abbd706f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:59.678445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-831643bf-c741-4d22-938a-bd4abbd706f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:59.764020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 822/1643] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:49:59.784719 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:49:59.788921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb7aa1-ce88-4e12-b62a-22372efeb127 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:59.790634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-23bb7aa1-ce88-4e12-b62a-22372efeb127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:59.814733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831643bf-c741-4d22-938a-bd4abbd706f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:59.816587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831643bf-c741-4d22-938a-bd4abbd706f5 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:59.830189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 822/1644] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:49:59.916414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb7aa1-ce88-4e12-b62a-22372efeb127 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:59.925039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de1644-4e1d-4d1f-833e-d6a4cbcd867c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:59.931619 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de1644-4e1d-4d1f-833e-d6a4cbcd867c service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:59.931774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831643bf-c741-4d22-938a-bd4abbd706f5 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:49:59.931922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de1644-4e1d-4d1f-833e-d6a4cbcd867c service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:59.932186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de1644-4e1d-4d1f-833e-d6a4cbcd867c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:59.933362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-831643bf-c741-4d22-938a-bd4abbd706f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:59.946253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831643bf-c741-4d22-938a-bd4abbd706f5 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:49:59.952384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef28084-17ae-4140-960e-ac81c6374cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:49:59.953247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef28084-17ae-4140-960e-ac81c6374cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:49:59.953498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef28084-17ae-4140-960e-ac81c6374cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:49:59.953782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef28084-17ae-4140-960e-ac81c6374cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:49:59.955636 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de1644-4e1d-4d1f-833e-d6a4cbcd867c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:49:59.956378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bef28084-17ae-4140-960e-ac81c6374cd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:49:59.959318 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de1644-4e1d-4d1f-833e-d6a4cbcd867c service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:49:59.978329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 823/1645] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:00.093532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef28084-17ae-4140-960e-ac81c6374cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:00.095309 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef28084-17ae-4140-960e-ac81c6374cd9 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:00.130793 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 823/1646] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:49:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:00.381526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de1644-4e1d-4d1f-833e-d6a4cbcd867c service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:00.383905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-24de1644-4e1d-4d1f-833e-d6a4cbcd867c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:00.408304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de1644-4e1d-4d1f-833e-d6a4cbcd867c service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:00.420705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:00.422853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:00.422853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:00.423223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] PATH_INFO: `/v3/users/948d7854fc784c4689b4c29ddcb13224` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:00.427762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=948d7854fc784c4689b4c29ddcb13224)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:00.432897 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:00.793786 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948d7854fc784c4689b4c29ddcb13224'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:00.811244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948d7854fc784c4689b4c29ddcb13224'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:00.837087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948d7854fc784c4689b4c29ddcb13224'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:00.919314 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948d7854fc784c4689b4c29ddcb13224'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:00.920785 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948d7854fc784c4689b4c29ddcb13224'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:00.933002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 824/1647] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:00 2022] DELETE /identity/v3/users/948d7854fc784c4689b4c29ddcb13224 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:00.937958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9c4f-bd4f-4ac6-a816-02195673fc11 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:00.939318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9c4f-bd4f-4ac6-a816-02195673fc11 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:00.939750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab9c4f-bd4f-4ac6-a816-02195673fc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:00.977211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7dab9c4f-bd4f-4ac6-a816-02195673fc11 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:00.979052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7dab9c4f-bd4f-4ac6-a816-02195673fc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:01.067214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:01.108683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 824/1648] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:01.122238 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:01.125010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:01.217883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:01.232018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc19709-9184-4fdf-be30-f6ca30ea9283 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:01.239930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3df6c-7428-4590-8be7-7dd485d8bd4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:01.241276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3df6c-7428-4590-8be7-7dd485d8bd4b service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:01.241803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3df6c-7428-4590-8be7-7dd485d8bd4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:01.242547 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3df6c-7428-4590-8be7-7dd485d8bd4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:01.259069 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3df6c-7428-4590-8be7-7dd485d8bd4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:01.264054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3df6c-7428-4590-8be7-7dd485d8bd4b service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:01.302252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 825/1649] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:01.659912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab9c4f-bd4f-4ac6-a816-02195673fc11 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:01.661420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7dab9c4f-bd4f-4ac6-a816-02195673fc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:01.679711 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab9c4f-bd4f-4ac6-a816-02195673fc11 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:01.697302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:01.698291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:01.698498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:01.698774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] PATH_INFO: `/v3/projects/67975f78146c41a9aecc4f09f993f6bc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:01.700841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67975f78146c41a9aecc4f09f993f6bc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:01.710733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: WARNING py.warnings [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 16:50:01.710733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: warnings.warn(msg) Jul 18 16:50:01.710733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: Jul 18 16:50:01.711998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:01.962360 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67975f78146c41a9aecc4f09f993f6bc', 'request_id': 'req-e6d3c766-e73f-4284-bf2e-15b6716b62d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:02.166709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67975f78146c41a9aecc4f09f993f6bc', 'request_id': 'req-e6d3c766-e73f-4284-bf2e-15b6716b62d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:02.166709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67975f78146c41a9aecc4f09f993f6bc', 'request_id': 'req-e6d3c766-e73f-4284-bf2e-15b6716b62d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:02.166709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] The token cache is being invalidate because project 67975f78146c41a9aecc4f09f993f6bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:02.166709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:02.167423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e6d3c766-e73f-4284-bf2e-15b6716b62d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:02.167423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 825/1650] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:01 2022] DELETE /identity/v3/projects/67975f78146c41a9aecc4f09f993f6bc => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:02.169159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367d3f0-af7c-44bc-a615-f70ff2a6c60d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:02.169648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367d3f0-af7c-44bc-a615-f70ff2a6c60d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:02.169648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367d3f0-af7c-44bc-a615-f70ff2a6c60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:02.219499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5367d3f0-af7c-44bc-a615-f70ff2a6c60d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:02.222597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5367d3f0-af7c-44bc-a615-f70ff2a6c60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:02.345597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad0cba-3007-4036-a22c-3c26435cdcaf None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:02.346206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad0cba-3007-4036-a22c-3c26435cdcaf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:02.346574 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad0cba-3007-4036-a22c-3c26435cdcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:02.348579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 826/1651] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:02.388226 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-94ad0cba-3007-4036-a22c-3c26435cdcaf None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:02.389495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-94ad0cba-3007-4036-a22c-3c26435cdcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:02.408430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 826/1652] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:02.413576 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367d3f0-af7c-44bc-a615-f70ff2a6c60d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:02.425590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367d3f0-af7c-44bc-a615-f70ff2a6c60d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:02.435444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eb318-f335-4c8e-a02f-7f0d4ac5917a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:02.436717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb318-f335-4c8e-a02f-7f0d4ac5917a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:02.437073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb318-f335-4c8e-a02f-7f0d4ac5917a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:02.437361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777eb318-f335-4c8e-a02f-7f0d4ac5917a admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:02.438189 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eb318-f335-4c8e-a02f-7f0d4ac5917a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:02.438501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eb318-f335-4c8e-a02f-7f0d4ac5917a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:02.440181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777eb318-f335-4c8e-a02f-7f0d4ac5917a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:02.455429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 827/1653] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:02.461292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad0cba-3007-4036-a22c-3c26435cdcaf None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:02.493198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad0cba-3007-4036-a22c-3c26435cdcaf None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:02.500405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29ca37-1413-46b7-b1a1-4ec8aaedceaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:02.503777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29ca37-1413-46b7-b1a1-4ec8aaedceaf admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:02.503777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29ca37-1413-46b7-b1a1-4ec8aaedceaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:02.503777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29ca37-1413-46b7-b1a1-4ec8aaedceaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:02.503777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29ca37-1413-46b7-b1a1-4ec8aaedceaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:02.505254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29ca37-1413-46b7-b1a1-4ec8aaedceaf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:02.936513 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eb318-f335-4c8e-a02f-7f0d4ac5917a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:02.940021 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-777eb318-f335-4c8e-a02f-7f0d4ac5917a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:02.973144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 827/1654] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:02 2022] POST /identity/v3/projects => generated 387 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:02.973144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29ca37-1413-46b7-b1a1-4ec8aaedceaf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:02.977170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ec29ca37-1413-46b7-b1a1-4ec8aaedceaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:02.979260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777eb318-f335-4c8e-a02f-7f0d4ac5917a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:03.184414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29ca37-1413-46b7-b1a1-4ec8aaedceaf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:03.188789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:03.189730 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:03.190016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:03.190293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] PATH_INFO: `/v3/users/330ff9790b3c4972b98521f1f6a52358` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:03.194049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] RBAC: Authorizing `identity:delete_user(user_id=330ff9790b3c4972b98521f1f6a52358)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:03.199015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:03.248771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:03.248771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:03.248771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:03.248771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] PATH_INFO: `/v3/projects/70335d12ffff4e399dd18661949c3b97` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:03.252762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70335d12ffff4e399dd18661949c3b97)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:03.254868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:03.476361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:03.499728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '330ff9790b3c4972b98521f1f6a52358'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:03.537068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '330ff9790b3c4972b98521f1f6a52358'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:03.539403 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70335d12ffff4e399dd18661949c3b97', 'request_id': 'req-597fff87-8c74-4089-a542-52a101834f27'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:03.547655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '330ff9790b3c4972b98521f1f6a52358'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:03.556003 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:03.570731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70335d12ffff4e399dd18661949c3b97', 'request_id': 'req-597fff87-8c74-4089-a542-52a101834f27'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:03.579873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70335d12ffff4e399dd18661949c3b97', 'request_id': 'req-597fff87-8c74-4089-a542-52a101834f27'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:03.629132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '330ff9790b3c4972b98521f1f6a52358'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:03.629971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '330ff9790b3c4972b98521f1f6a52358'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:03.651888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 828/1655] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:02 2022] DELETE /identity/v3/users/330ff9790b3c4972b98521f1f6a52358 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:03.655555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:03.666274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:03.722137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] The token cache is being invalidate because project 70335d12ffff4e399dd18661949c3b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:03.722437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:03.725055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-597fff87-8c74-4089-a542-52a101834f27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:03.728457 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 828/1656] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:02 2022] DELETE /identity/v3/projects/70335d12ffff4e399dd18661949c3b97 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:03.731073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b021350-9361-45be-bcae-0ab2e9fee000 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:03.731406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b021350-9361-45be-bcae-0ab2e9fee000 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:03.736076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b021350-9361-45be-bcae-0ab2e9fee000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:03.808634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0b021350-9361-45be-bcae-0ab2e9fee000 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:03.810142 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0b021350-9361-45be-bcae-0ab2e9fee000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:03.912477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525e59f-3428-45a2-83a0-694e8100460b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:03.929985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40718919-f409-430b-a23e-3f1593da16eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:03.930961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718919-f409-430b-a23e-3f1593da16eb admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:03.931260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718919-f409-430b-a23e-3f1593da16eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:03.931661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718919-f409-430b-a23e-3f1593da16eb admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:03.932453 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40718919-f409-430b-a23e-3f1593da16eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:03.938851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40718919-f409-430b-a23e-3f1593da16eb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:03.984833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 829/1657] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:03.989653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b021350-9361-45be-bcae-0ab2e9fee000 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:03.996178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0b021350-9361-45be-bcae-0ab2e9fee000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:04.011541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b021350-9361-45be-bcae-0ab2e9fee000 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:04.022124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:04.025611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:04.025611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:04.025611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] PATH_INFO: `/v3/users/9693a07b07bf43d6b59090150df8b377` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:04.025611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9693a07b07bf43d6b59090150df8b377)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:04.028341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:04.103282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 829/1658] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:03 2022] POST /identity/v3/users => generated 294 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:04.119946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40718919-f409-430b-a23e-3f1593da16eb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:04.125256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-40718919-f409-430b-a23e-3f1593da16eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:04.239732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40718919-f409-430b-a23e-3f1593da16eb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:04.250865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296085a-8989-4cb6-9f45-491a68e56477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:04.252088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296085a-8989-4cb6-9f45-491a68e56477 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:04.252560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296085a-8989-4cb6-9f45-491a68e56477 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:04.253181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296085a-8989-4cb6-9f45-491a68e56477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:04.274275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296085a-8989-4cb6-9f45-491a68e56477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:04.276186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296085a-8989-4cb6-9f45-491a68e56477 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:04.292864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 830/1659] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:04.294165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296085a-8989-4cb6-9f45-491a68e56477 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:04.299263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c296085a-8989-4cb6-9f45-491a68e56477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:04.348686 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296085a-8989-4cb6-9f45-491a68e56477 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:04.363819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9693a07b07bf43d6b59090150df8b377'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:04.372326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06c153-740b-40ca-b52a-9da28d7a6778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:04.373432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06c153-740b-40ca-b52a-9da28d7a6778 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:04.373890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06c153-740b-40ca-b52a-9da28d7a6778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:04.374538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06c153-740b-40ca-b52a-9da28d7a6778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:04.375447 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06c153-740b-40ca-b52a-9da28d7a6778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:04.378469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06c153-740b-40ca-b52a-9da28d7a6778 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:04.400846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 831/1660] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:04.405543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9693a07b07bf43d6b59090150df8b377'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:04.405659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06c153-740b-40ca-b52a-9da28d7a6778 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:04.444387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06c153-740b-40ca-b52a-9da28d7a6778 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:04.454239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddc522-0448-44fe-9ce7-6c9516ba242b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:04.459199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddc522-0448-44fe-9ce7-6c9516ba242b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:04.460006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddc522-0448-44fe-9ce7-6c9516ba242b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:04.460410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddc522-0448-44fe-9ce7-6c9516ba242b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:04.461481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddc522-0448-44fe-9ce7-6c9516ba242b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:04.463506 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddc522-0448-44fe-9ce7-6c9516ba242b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:04.471406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9693a07b07bf43d6b59090150df8b377'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:04.497410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 832/1661] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:04.503535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddc522-0448-44fe-9ce7-6c9516ba242b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:04.523946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddc522-0448-44fe-9ce7-6c9516ba242b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:04.538380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffde40-f6f8-4586-8e3a-5b5082529853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:04.539726 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffde40-f6f8-4586-8e3a-5b5082529853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:04.540290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffde40-f6f8-4586-8e3a-5b5082529853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:04.541628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffde40-f6f8-4586-8e3a-5b5082529853 admin admin] PATH_INFO: `/v3/projects/5e64285d36bd45f288981e5e80f96372/users/3e047f9749954ae19e13a5cc4743f338/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:04.569217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffde40-f6f8-4586-8e3a-5b5082529853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e64285d36bd45f288981e5e80f96372, user_id=3e047f9749954ae19e13a5cc4743f338, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:04.572008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9693a07b07bf43d6b59090150df8b377'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:04.573310 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9693a07b07bf43d6b59090150df8b377'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:04.577835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffde40-f6f8-4586-8e3a-5b5082529853 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:04.582611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 830/1662] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:03 2022] DELETE /identity/v3/users/9693a07b07bf43d6b59090150df8b377 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:04.627132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:04.632100 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:04.704160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 833/1663] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:04 2022] PUT /identity/v3/projects/5e64285d36bd45f288981e5e80f96372/users/3e047f9749954ae19e13a5cc4743f338/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:04.704160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f604ae1-c450-483e-ac3b-c54beb41654b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:04.704160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f604ae1-c450-483e-ac3b-c54beb41654b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:04.704160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f604ae1-c450-483e-ac3b-c54beb41654b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:04.795856 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5f604ae1-c450-483e-ac3b-c54beb41654b None None] MFA Rules not processed for user `3e047f9749954ae19e13a5cc4743f338`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:04.806203 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5f604ae1-c450-483e-ac3b-c54beb41654b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:04.823076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:04.873526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3b2df-224b-4fba-9e7c-57f91a270106 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:04.986455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 834/1664] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:04.990258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f604ae1-c450-483e-ac3b-c54beb41654b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:04.995007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5f604ae1-c450-483e-ac3b-c54beb41654b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:05.008651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239821a-60d4-4999-869d-1fd778ebd447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:05.012341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239821a-60d4-4999-869d-1fd778ebd447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:05.012819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239821a-60d4-4999-869d-1fd778ebd447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:05.013196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239821a-60d4-4999-869d-1fd778ebd447 admin admin] PATH_INFO: `/v3/users/f169cd11553546adb4aceb8c0672a084` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:05.016590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239821a-60d4-4999-869d-1fd778ebd447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f169cd11553546adb4aceb8c0672a084)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:05.019612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239821a-60d4-4999-869d-1fd778ebd447 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:05.039007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f604ae1-c450-483e-ac3b-c54beb41654b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:05.045452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adc025-9bc5-4736-a1ec-0cea3870d84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:05.046188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adc025-9bc5-4736-a1ec-0cea3870d84f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:05.046816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adc025-9bc5-4736-a1ec-0cea3870d84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:05.047749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15adc025-9bc5-4736-a1ec-0cea3870d84f admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:05.048511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15adc025-9bc5-4736-a1ec-0cea3870d84f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:05.051225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15adc025-9bc5-4736-a1ec-0cea3870d84f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:05.206893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 835/1665] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:04 2022] POST /identity/v3/projects => generated 387 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:05.209032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adc025-9bc5-4736-a1ec-0cea3870d84f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:05.323924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f239821a-60d4-4999-869d-1fd778ebd447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f169cd11553546adb4aceb8c0672a084'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:05.352511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15adc025-9bc5-4736-a1ec-0cea3870d84f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:05.352790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f239821a-60d4-4999-869d-1fd778ebd447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f169cd11553546adb4aceb8c0672a084'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:05.355612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b2e45-1775-4044-bf25-364f2aedea06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:05.356381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b2e45-1775-4044-bf25-364f2aedea06 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:05.356738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b2e45-1775-4044-bf25-364f2aedea06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:05.357114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b2e45-1775-4044-bf25-364f2aedea06 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:05.358373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b2e45-1775-4044-bf25-364f2aedea06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:05.360966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b2e45-1775-4044-bf25-364f2aedea06 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:05.386048 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f239821a-60d4-4999-869d-1fd778ebd447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f169cd11553546adb4aceb8c0672a084'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:05.511858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f239821a-60d4-4999-869d-1fd778ebd447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f169cd11553546adb4aceb8c0672a084'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:05.519139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-f239821a-60d4-4999-869d-1fd778ebd447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f169cd11553546adb4aceb8c0672a084'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:05.535687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 836/1666] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:05 2022] POST /identity/v3/users => generated 300 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:50:05.537851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b2e45-1775-4044-bf25-364f2aedea06 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:05.541799 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b01b2e45-1775-4044-bf25-364f2aedea06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:05.553525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 831/1667] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:04 2022] DELETE /identity/v3/users/f169cd11553546adb4aceb8c0672a084 => generated 0 bytes in 928 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:05.563633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6662ee-b23c-4052-9823-d4b87d31854b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:05.564174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6662ee-b23c-4052-9823-d4b87d31854b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:05.565323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6662ee-b23c-4052-9823-d4b87d31854b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:05.577928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b2e45-1775-4044-bf25-364f2aedea06 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:05.626385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:05.672237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3f6662ee-b23c-4052-9823-d4b87d31854b None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:05.677339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3f6662ee-b23c-4052-9823-d4b87d31854b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:05.763698 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60b67e-78c9-4c79-afff-4504a1fb405c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:05.765211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60b67e-78c9-4c79-afff-4504a1fb405c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:05.765656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60b67e-78c9-4c79-afff-4504a1fb405c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:05.766825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60b67e-78c9-4c79-afff-4504a1fb405c admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:05.767816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60b67e-78c9-4c79-afff-4504a1fb405c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:05.770681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60b67e-78c9-4c79-afff-4504a1fb405c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:05.788812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 837/1668] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:05.789780 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 832/1669] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:05.798780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60b67e-78c9-4c79-afff-4504a1fb405c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:05.819634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6662ee-b23c-4052-9823-d4b87d31854b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:05.821846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3f6662ee-b23c-4052-9823-d4b87d31854b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:05.830028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60b67e-78c9-4c79-afff-4504a1fb405c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:05.837347 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78803785-7832-488e-990f-016323a78bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:05.840122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78803785-7832-488e-990f-016323a78bad admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:05.840757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78803785-7832-488e-990f-016323a78bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:05.841187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78803785-7832-488e-990f-016323a78bad admin admin] PATH_INFO: `/v3/projects/bd5f3b0f6d7944bbadb6c86a2000c13e/users/7060f13071764eb19fe4858f3ee68011/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:05.866583 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78803785-7832-488e-990f-016323a78bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5f3b0f6d7944bbadb6c86a2000c13e, user_id=7060f13071764eb19fe4858f3ee68011, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:05.876914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78803785-7832-488e-990f-016323a78bad admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:05.928266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6662ee-b23c-4052-9823-d4b87d31854b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:05.932455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a5669-1c45-4297-9e87-6b32ca87b09a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:05.933528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a5669-1c45-4297-9e87-6b32ca87b09a service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:05.933528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a5669-1c45-4297-9e87-6b32ca87b09a service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:05.938488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a5669-1c45-4297-9e87-6b32ca87b09a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:05.960160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a5669-1c45-4297-9e87-6b32ca87b09a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:05.966990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a5669-1c45-4297-9e87-6b32ca87b09a service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:05.983939 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 838/1670] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:05 2022] PUT /identity/v3/projects/bd5f3b0f6d7944bbadb6c86a2000c13e/users/7060f13071764eb19fe4858f3ee68011/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:05.984919 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78803785-7832-488e-990f-016323a78bad admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:05.987868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-78803785-7832-488e-990f-016323a78bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:06.000753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 833/1671] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:06.001707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a5669-1c45-4297-9e87-6b32ca87b09a service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:06.022989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a5669-1c45-4297-9e87-6b32ca87b09a service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:06.149745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727b702-1675-4411-906f-8dd62208b632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:06.150479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727b702-1675-4411-906f-8dd62208b632 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:06.150720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727b702-1675-4411-906f-8dd62208b632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:06.150998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5727b702-1675-4411-906f-8dd62208b632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:06.151742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727b702-1675-4411-906f-8dd62208b632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:06.154180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5727b702-1675-4411-906f-8dd62208b632 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:06.179112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 834/1672] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:06.179294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78803785-7832-488e-990f-016323a78bad admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:06.199799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727b702-1675-4411-906f-8dd62208b632 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:06.200171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:06.200171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:06.200171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:06.200171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] PATH_INFO: `/v3/projects/14f3713ecf7246a9a286e57577014bdc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:06.200503 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f3713ecf7246a9a286e57577014bdc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:06.204728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:06.219370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5727b702-1675-4411-906f-8dd62208b632 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:06.231517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b396e43-f45a-412c-aa86-d23054ce5b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:06.231517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b396e43-f45a-412c-aa86-d23054ce5b2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:06.231517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b396e43-f45a-412c-aa86-d23054ce5b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:06.231517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b396e43-f45a-412c-aa86-d23054ce5b2e admin admin] PATH_INFO: `/v3/projects/bd5f3b0f6d7944bbadb6c86a2000c13e/users/7060f13071764eb19fe4858f3ee68011/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:06.232340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b396e43-f45a-412c-aa86-d23054ce5b2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd5f3b0f6d7944bbadb6c86a2000c13e, user_id=7060f13071764eb19fe4858f3ee68011, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:06.235411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b396e43-f45a-412c-aa86-d23054ce5b2e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:06.249130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 835/1673] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:06 2022] PUT /identity/v3/projects/bd5f3b0f6d7944bbadb6c86a2000c13e/users/7060f13071764eb19fe4858f3ee68011/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:06.260490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaa322-b878-46f2-ac9c-dc9d2440f79a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:06.261060 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaa322-b878-46f2-ac9c-dc9d2440f79a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:06.261517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaa322-b878-46f2-ac9c-dc9d2440f79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:06.307764 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4afaa322-b878-46f2-ac9c-dc9d2440f79a None None] MFA Rules not processed for user `7060f13071764eb19fe4858f3ee68011`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:06.309040 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4afaa322-b878-46f2-ac9c-dc9d2440f79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:06.441149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f3713ecf7246a9a286e57577014bdc', 'request_id': 'req-35950df6-286a-4966-96b2-8664dd6c70ed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:06.476688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 836/1674] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:06.491576 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afaa322-b878-46f2-ac9c-dc9d2440f79a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:06.493554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4afaa322-b878-46f2-ac9c-dc9d2440f79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:06.554701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f3713ecf7246a9a286e57577014bdc', 'request_id': 'req-35950df6-286a-4966-96b2-8664dd6c70ed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:06.571426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f3713ecf7246a9a286e57577014bdc', 'request_id': 'req-35950df6-286a-4966-96b2-8664dd6c70ed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:06.612452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] The token cache is being invalidate because project 14f3713ecf7246a9a286e57577014bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:06.617769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:06.621156 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:06.623822 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 839/1675] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:05 2022] DELETE /identity/v3/projects/14f3713ecf7246a9a286e57577014bdc => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:06.626939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afaa322-b878-46f2-ac9c-dc9d2440f79a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:06.627398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:06.635231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:06.743352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0ce10-0205-4b27-9b1d-31af58647b26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:06.744832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0ce10-0205-4b27-9b1d-31af58647b26 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:06.744832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0ce10-0205-4b27-9b1d-31af58647b26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:06.744832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0ce10-0205-4b27-9b1d-31af58647b26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:06.748433 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5dc0ce10-0205-4b27-9b1d-31af58647b26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:06.921218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35950df6-286a-4966-96b2-8664dd6c70ed admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:06.951576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:06.952483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:06.952849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:06.953069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] PATH_INFO: `/v3/projects/44f98c5e4b8645048b138492c4d21992` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:06.960941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f98c5e4b8645048b138492c4d21992)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:06.980826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:06.983622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:07.076244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0ce10-0205-4b27-9b1d-31af58647b26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:07.078936 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0ce10-0205-4b27-9b1d-31af58647b26 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:07.120928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 837/1676] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:07.122616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1939e-e319-4dcb-9b15-bb0c22503fd3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:07.125637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1939e-e319-4dcb-9b15-bb0c22503fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:07.126076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1939e-e319-4dcb-9b15-bb0c22503fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:07.173417 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-99d1939e-e319-4dcb-9b15-bb0c22503fd3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:07.175953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-99d1939e-e319-4dcb-9b15-bb0c22503fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:07.248796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 838/1677] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:07.250907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1939e-e319-4dcb-9b15-bb0c22503fd3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:07.253320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-99d1939e-e319-4dcb-9b15-bb0c22503fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:07.278229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1939e-e319-4dcb-9b15-bb0c22503fd3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:07.294361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f98c5e4b8645048b138492c4d21992', 'request_id': 'req-ad927218-b07e-44c0-b3c0-bd5dfd263075'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:07.386738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f98c5e4b8645048b138492c4d21992', 'request_id': 'req-ad927218-b07e-44c0-b3c0-bd5dfd263075'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:07.403310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f98c5e4b8645048b138492c4d21992', 'request_id': 'req-ad927218-b07e-44c0-b3c0-bd5dfd263075'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:07.428006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] The token cache is being invalidate because project 44f98c5e4b8645048b138492c4d21992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:07.428006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:07.432019 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ad927218-b07e-44c0-b3c0-bd5dfd263075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:07.433722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 840/1678] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:06 2022] DELETE /identity/v3/projects/44f98c5e4b8645048b138492c4d21992 => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:07.443565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583266a4-9768-4793-8a18-86309a4b8f0a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:07.444258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583266a4-9768-4793-8a18-86309a4b8f0a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:07.444258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583266a4-9768-4793-8a18-86309a4b8f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:07.455783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:07.457382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:07.457382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:07.457572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] PATH_INFO: `/v3/users/3e047f9749954ae19e13a5cc4743f338` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:07.461667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e047f9749954ae19e13a5cc4743f338)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:07.473193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:07.502837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-583266a4-9768-4793-8a18-86309a4b8f0a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:07.505854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:07.507007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-583266a4-9768-4793-8a18-86309a4b8f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:07.531988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 841/1679] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:07.532599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583266a4-9768-4793-8a18-86309a4b8f0a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:07.534015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-583266a4-9768-4793-8a18-86309a4b8f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:07.572484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583266a4-9768-4793-8a18-86309a4b8f0a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:07.601038 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41573812-7468-441d-b605-f111ad75c0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:07.601952 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41573812-7468-441d-b605-f111ad75c0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:07.602587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41573812-7468-441d-b605-f111ad75c0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:07.602587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41573812-7468-441d-b605-f111ad75c0ca admin admin] PATH_INFO: `/v3/users/b95c1b4fcf1f4daa8837757d5169d36a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:07.603762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41573812-7468-441d-b605-f111ad75c0ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b95c1b4fcf1f4daa8837757d5169d36a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:07.605807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41573812-7468-441d-b605-f111ad75c0ca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:07.857380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e047f9749954ae19e13a5cc4743f338'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:07.886423 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e047f9749954ae19e13a5cc4743f338'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:07.946944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e047f9749954ae19e13a5cc4743f338'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:07.976791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-41573812-7468-441d-b605-f111ad75c0ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b95c1b4fcf1f4daa8837757d5169d36a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.003764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-41573812-7468-441d-b605-f111ad75c0ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b95c1b4fcf1f4daa8837757d5169d36a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.014391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-41573812-7468-441d-b605-f111ad75c0ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b95c1b4fcf1f4daa8837757d5169d36a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.076637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e047f9749954ae19e13a5cc4743f338'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.080379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e047f9749954ae19e13a5cc4743f338'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.095419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 839/1680] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:07 2022] DELETE /identity/v3/users/3e047f9749954ae19e13a5cc4743f338 => generated 0 bytes in 846 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:08.100152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:08.105350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:08.166973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-41573812-7468-441d-b605-f111ad75c0ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b95c1b4fcf1f4daa8837757d5169d36a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.167855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-41573812-7468-441d-b605-f111ad75c0ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b95c1b4fcf1f4daa8837757d5169d36a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.181517 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 842/1681] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:07 2022] DELETE /identity/v3/users/b95c1b4fcf1f4daa8837757d5169d36a => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:08.182989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a9ce-8c26-4b70-9db7-0ebcd5b54281 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:08.183282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a9ce-8c26-4b70-9db7-0ebcd5b54281 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:08.183562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49a9ce-8c26-4b70-9db7-0ebcd5b54281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:08.227876 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9f49a9ce-8c26-4b70-9db7-0ebcd5b54281 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:08.236440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9f49a9ce-8c26-4b70-9db7-0ebcd5b54281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:08.322539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246142d-f607-402c-bdf3-abd63a755a9a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:08.403589 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 843/1682] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:08.404442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49a9ce-8c26-4b70-9db7-0ebcd5b54281 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:08.405956 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9f49a9ce-8c26-4b70-9db7-0ebcd5b54281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:08.442132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49a9ce-8c26-4b70-9db7-0ebcd5b54281 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:08.466896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:08.466896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:08.466896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:08.467842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] PATH_INFO: `/v3/projects/c956cdbbf8d14dfe89b00a63a98dcc2e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:08.471814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c956cdbbf8d14dfe89b00a63a98dcc2e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:08.484747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:08.505831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc41454-9039-4705-b888-775503d199ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:08.506543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41454-9039-4705-b888-775503d199ee admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:08.509947 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41454-9039-4705-b888-775503d199ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:08.510351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41454-9039-4705-b888-775503d199ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:08.511696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc41454-9039-4705-b888-775503d199ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:08.512018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc41454-9039-4705-b888-775503d199ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:08.518839 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc41454-9039-4705-b888-775503d199ee admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:08.554516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 840/1683] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:08.554516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc41454-9039-4705-b888-775503d199ee admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:08.559355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5fc41454-9039-4705-b888-775503d199ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:08.693951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c956cdbbf8d14dfe89b00a63a98dcc2e', 'request_id': 'req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.727649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc41454-9039-4705-b888-775503d199ee admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:08.794788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c956cdbbf8d14dfe89b00a63a98dcc2e', 'request_id': 'req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.805642 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c956cdbbf8d14dfe89b00a63a98dcc2e', 'request_id': 'req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.828925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] The token cache is being invalidate because project c956cdbbf8d14dfe89b00a63a98dcc2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:08.828925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.829756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:08.833198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 844/1684] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:08 2022] DELETE /identity/v3/projects/c956cdbbf8d14dfe89b00a63a98dcc2e => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:08.833198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:08.833730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e953e97-0b0a-480f-b773-965c5d6732b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:08.833948 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:08.834403 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e953e97-0b0a-480f-b773-965c5d6732b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:08.834690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e953e97-0b0a-480f-b773-965c5d6732b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:08.834964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e953e97-0b0a-480f-b773-965c5d6732b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:08.839326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5e953e97-0b0a-480f-b773-965c5d6732b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:08.869018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:09.046271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d1d2b-1e0f-46e6-af7b-b9e0a8fba5d8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:09.064695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e1ca0-6cfc-40d1-9751-a6083f418b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:09.065248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e953e97-0b0a-480f-b773-965c5d6732b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:09.066464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e1ca0-6cfc-40d1-9751-a6083f418b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:09.070598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e1ca0-6cfc-40d1-9751-a6083f418b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:09.072066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e1ca0-6cfc-40d1-9751-a6083f418b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:09.072066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e1ca0-6cfc-40d1-9751-a6083f418b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:09.078371 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e953e97-0b0a-480f-b773-965c5d6732b9 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:09.084458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e1ca0-6cfc-40d1-9751-a6083f418b13 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:09.097128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 841/1685] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:09.099321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f8fff-2186-4ee3-92e9-733f699a61bb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:09.099700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f8fff-2186-4ee3-92e9-733f699a61bb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:09.100193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f8fff-2186-4ee3-92e9-733f699a61bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:09.143084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5d9f8fff-2186-4ee3-92e9-733f699a61bb None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:09.144334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5d9f8fff-2186-4ee3-92e9-733f699a61bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:09.171216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 842/1686] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:09.179229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb92a47-b556-4a97-a32a-6ad20804ac05 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:09.179518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb92a47-b556-4a97-a32a-6ad20804ac05 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:09.179798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb92a47-b556-4a97-a32a-6ad20804ac05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:09.211285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-eeb92a47-b556-4a97-a32a-6ad20804ac05 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:09.212796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-eeb92a47-b556-4a97-a32a-6ad20804ac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:09.226572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 843/1687] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:09.231752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb92a47-b556-4a97-a32a-6ad20804ac05 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:09.238666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 845/1688] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:08 2022] POST /identity/v3/projects => generated 373 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:09.240463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e1ca0-6cfc-40d1-9751-a6083f418b13 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:09.259890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb92a47-b556-4a97-a32a-6ad20804ac05 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:09.368349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:09.401779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e1ca0-6cfc-40d1-9751-a6083f418b13 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:09.416277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9209695-aa3a-4bd1-a0da-8476cd91d1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:09.417576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9209695-aa3a-4bd1-a0da-8476cd91d1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:09.417879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9209695-aa3a-4bd1-a0da-8476cd91d1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:09.418159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9209695-aa3a-4bd1-a0da-8476cd91d1a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:09.418867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9209695-aa3a-4bd1-a0da-8476cd91d1a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:09.421584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9209695-aa3a-4bd1-a0da-8476cd91d1a7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:09.458965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f39ec-81ff-4b70-9d7b-b0999e897a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:09.461254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f39ec-81ff-4b70-9d7b-b0999e897a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:09.463293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f39ec-81ff-4b70-9d7b-b0999e897a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:09.466057 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f39ec-81ff-4b70-9d7b-b0999e897a95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:09.466941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f39ec-81ff-4b70-9d7b-b0999e897a95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:09.468811 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f39ec-81ff-4b70-9d7b-b0999e897a95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:09.472621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f39ec-81ff-4b70-9d7b-b0999e897a95 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:09.525233 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 844/1689] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:09.538150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f39ec-81ff-4b70-9d7b-b0999e897a95 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:09.556031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f39ec-81ff-4b70-9d7b-b0999e897a95 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:09.563196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b580a86-ff67-44b0-85f0-36d8ddcc856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:09.563990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b580a86-ff67-44b0-85f0-36d8ddcc856d admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:09.578702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 846/1690] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:09 2022] POST /identity/v3/users => generated 287 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:09.595112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b580a86-ff67-44b0-85f0-36d8ddcc856d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:09.595539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b580a86-ff67-44b0-85f0-36d8ddcc856d admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:09.598854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9209695-aa3a-4bd1-a0da-8476cd91d1a7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:09.607278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b580a86-ff67-44b0-85f0-36d8ddcc856d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:09.613802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b580a86-ff67-44b0-85f0-36d8ddcc856d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:09.634054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9209695-aa3a-4bd1-a0da-8476cd91d1a7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:09.650939 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9db3bf-7284-46e8-96ae-9b24922b97d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:09.651808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9db3bf-7284-46e8-96ae-9b24922b97d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:09.652097 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9db3bf-7284-46e8-96ae-9b24922b97d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:09.652370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9db3bf-7284-46e8-96ae-9b24922b97d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:09.656158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9db3bf-7284-46e8-96ae-9b24922b97d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:09.660136 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9db3bf-7284-46e8-96ae-9b24922b97d2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:09.674102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 847/1691] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:09.679829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9db3bf-7284-46e8-96ae-9b24922b97d2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:09.703191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9db3bf-7284-46e8-96ae-9b24922b97d2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:09.723124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5f487-95a5-4cf7-b58a-64a02b9f565a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:09.724034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5f487-95a5-4cf7-b58a-64a02b9f565a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:09.724900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5f487-95a5-4cf7-b58a-64a02b9f565a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:09.725192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5f487-95a5-4cf7-b58a-64a02b9f565a admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:09.726052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5f487-95a5-4cf7-b58a-64a02b9f565a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:09.730924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5f487-95a5-4cf7-b58a-64a02b9f565a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:09.756586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 845/1692] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:09 2022] POST /identity/v3/projects => generated 377 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:09.761334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b580a86-ff67-44b0-85f0-36d8ddcc856d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:09.768272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 848/1693] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:09.773600 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5f487-95a5-4cf7-b58a-64a02b9f565a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:09.802164 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5f487-95a5-4cf7-b58a-64a02b9f565a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:09.891565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b580a86-ff67-44b0-85f0-36d8ddcc856d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:09.897579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81b264-0042-405c-9043-1a5e1d75c550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:09.898299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81b264-0042-405c-9043-1a5e1d75c550 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:09.899046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81b264-0042-405c-9043-1a5e1d75c550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:09.899046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81b264-0042-405c-9043-1a5e1d75c550 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:09.899548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd81b264-0042-405c-9043-1a5e1d75c550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:09.902262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd81b264-0042-405c-9043-1a5e1d75c550 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:09.968996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a20818-60bf-46b9-87f4-14aa95fbc95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:09.970238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a20818-60bf-46b9-87f4-14aa95fbc95e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:09.970830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a20818-60bf-46b9-87f4-14aa95fbc95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:09.971117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a20818-60bf-46b9-87f4-14aa95fbc95e admin admin] PATH_INFO: `/v3/projects/f3a4ff367f144d8186422c24642d8038/users/8b2183a1b4c6417d9e0ceb56ddfad9a3/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:10.012087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a20818-60bf-46b9-87f4-14aa95fbc95e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a4ff367f144d8186422c24642d8038, user_id=8b2183a1b4c6417d9e0ceb56ddfad9a3, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:10.015753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a20818-60bf-46b9-87f4-14aa95fbc95e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:10.048771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 846/1694] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:09 2022] POST /identity/v3/users => generated 289 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:10.048771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81b264-0042-405c-9043-1a5e1d75c550 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:10.083846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81b264-0042-405c-9043-1a5e1d75c550 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:10.119867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09c188-e3a7-4d4d-9aa9-886c93b330f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:10.120577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09c188-e3a7-4d4d-9aa9-886c93b330f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:10.120867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09c188-e3a7-4d4d-9aa9-886c93b330f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:10.121175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09c188-e3a7-4d4d-9aa9-886c93b330f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:10.121931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09c188-e3a7-4d4d-9aa9-886c93b330f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:10.141154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 849/1695] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:09 2022] PUT /identity/v3/projects/f3a4ff367f144d8186422c24642d8038/users/8b2183a1b4c6417d9e0ceb56ddfad9a3/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:10.141342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09c188-e3a7-4d4d-9aa9-886c93b330f3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:10.149559 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e348-415e-4f06-8bfa-27bf5c71a295 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:10.149857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e348-415e-4f06-8bfa-27bf5c71a295 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:10.150143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e348-415e-4f06-8bfa-27bf5c71a295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:10.167293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 847/1696] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:10.173054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09c188-e3a7-4d4d-9aa9-886c93b330f3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:10.197767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09c188-e3a7-4d4d-9aa9-886c93b330f3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:10.215215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2785e348-415e-4f06-8bfa-27bf5c71a295 None None] MFA Rules not processed for user `8b2183a1b4c6417d9e0ceb56ddfad9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:10.217109 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2785e348-415e-4f06-8bfa-27bf5c71a295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:10.344037 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db473c74-e15b-4f36-b222-8f0385568d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:10.345243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db473c74-e15b-4f36-b222-8f0385568d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:10.345753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db473c74-e15b-4f36-b222-8f0385568d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:10.346199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db473c74-e15b-4f36-b222-8f0385568d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:10.347147 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db473c74-e15b-4f36-b222-8f0385568d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:10.350137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db473c74-e15b-4f36-b222-8f0385568d42 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:10.368820 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 848/1697] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:10.369713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db473c74-e15b-4f36-b222-8f0385568d42 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:10.375128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-db473c74-e15b-4f36-b222-8f0385568d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:10.402652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db473c74-e15b-4f36-b222-8f0385568d42 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:10.412274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 850/1698] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:10.413315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785e348-415e-4f06-8bfa-27bf5c71a295 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:10.417767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:10.418767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:10.419312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:10.419725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] PATH_INFO: `/v3/projects/d0ad5e03cebc44d6b2df0d30505083b4` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:10.421236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ad5e03cebc44d6b2df0d30505083b4)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:10.424036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:10.445520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785e348-415e-4f06-8bfa-27bf5c71a295 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:10.468469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdff5f-21c1-40a4-b2ac-d49f34dd7632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:10.470535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdff5f-21c1-40a4-b2ac-d49f34dd7632 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:10.470535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdff5f-21c1-40a4-b2ac-d49f34dd7632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:10.471029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdff5f-21c1-40a4-b2ac-d49f34dd7632 admin admin] PATH_INFO: `/v3/projects/68ec43e2be644aadbbeaab87220a993c/users/e2ca05736b6e46a0b3d3094981ce22d8/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:10.511209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdff5f-21c1-40a4-b2ac-d49f34dd7632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ec43e2be644aadbbeaab87220a993c, user_id=e2ca05736b6e46a0b3d3094981ce22d8, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:10.514361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdff5f-21c1-40a4-b2ac-d49f34dd7632 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:10.630648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 851/1699] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:10 2022] PUT /identity/v3/projects/68ec43e2be644aadbbeaab87220a993c/users/e2ca05736b6e46a0b3d3094981ce22d8/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:10.631343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdff5f-21c1-40a4-b2ac-d49f34dd7632 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:10.659126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ad5e03cebc44d6b2df0d30505083b4', 'request_id': 'req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:10.664850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:10.673581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ad5e03cebc44d6b2df0d30505083b4', 'request_id': 'req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:10.687131 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ad5e03cebc44d6b2df0d30505083b4', 'request_id': 'req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:10.785216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdff5f-21c1-40a4-b2ac-d49f34dd7632 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:10.818146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:10.824761 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] The token cache is being invalidate because project d0ad5e03cebc44d6b2df0d30505083b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:10.831481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:10.831481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6868cc5f-f3ff-4018-bea4-aed3fbdc211d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:10.831481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 849/1700] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:10 2022] DELETE /identity/v3/projects/d0ad5e03cebc44d6b2df0d30505083b4 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:10.834495 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9eb5d-179b-4804-a450-5252b554299d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:10.850359 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9eb5d-179b-4804-a450-5252b554299d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:10.857201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9eb5d-179b-4804-a450-5252b554299d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:10.978538 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-aeb9eb5d-179b-4804-a450-5252b554299d None None] MFA Rules not processed for user `f2d1f86034b94e289bb1fc754f02f4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:10.979816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-aeb9eb5d-179b-4804-a450-5252b554299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:10.985211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061f697-a1bc-4403-b1a7-dc8ad6d809df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:10.992520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f697-a1bc-4403-b1a7-dc8ad6d809df admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:10.992520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f697-a1bc-4403-b1a7-dc8ad6d809df admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:10.992520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f697-a1bc-4403-b1a7-dc8ad6d809df admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:10.992520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061f697-a1bc-4403-b1a7-dc8ad6d809df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:10.995567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061f697-a1bc-4403-b1a7-dc8ad6d809df admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:11.116202 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 852/1701] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:10 2022] POST /identity/v3/projects => generated 371 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:11.120071 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774337ed-adc9-48b6-b02c-0f1272a4719b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:11.121100 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774337ed-adc9-48b6-b02c-0f1272a4719b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:11.126784 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774337ed-adc9-48b6-b02c-0f1272a4719b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:11.157105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 850/1702] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:11.158673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d3a3f-0ba3-4a35-a55f-c5abe3903615 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:11.162912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d3a3f-0ba3-4a35-a55f-c5abe3903615 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:11.162912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d3a3f-0ba3-4a35-a55f-c5abe3903615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:11.209482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-570d3a3f-0ba3-4a35-a55f-c5abe3903615 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:11.211792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-570d3a3f-0ba3-4a35-a55f-c5abe3903615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:11.214505 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-774337ed-adc9-48b6-b02c-0f1272a4719b None None] MFA Rules not processed for user `e2ca05736b6e46a0b3d3094981ce22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:11.218486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-774337ed-adc9-48b6-b02c-0f1272a4719b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:11.386698 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 853/1703] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:11.397492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774337ed-adc9-48b6-b02c-0f1272a4719b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:11.400427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-774337ed-adc9-48b6-b02c-0f1272a4719b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:11.452767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 851/1704] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:11.453655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d3a3f-0ba3-4a35-a55f-c5abe3903615 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:11.459088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-570d3a3f-0ba3-4a35-a55f-c5abe3903615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:11.474111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:11.492025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d3a3f-0ba3-4a35-a55f-c5abe3903615 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:11.516857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a1f1b-5998-474f-8d16-1fd33b3e356b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:11.517888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1f1b-5998-474f-8d16-1fd33b3e356b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:11.518309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1f1b-5998-474f-8d16-1fd33b3e356b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:11.518890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a1f1b-5998-474f-8d16-1fd33b3e356b admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:11.519772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a1f1b-5998-474f-8d16-1fd33b3e356b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:11.522347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a1f1b-5998-474f-8d16-1fd33b3e356b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:11.545063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774337ed-adc9-48b6-b02c-0f1272a4719b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:11.554555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:11.555522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:11.557550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:11.557938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] PATH_INFO: `/v3/projects/5e64285d36bd45f288981e5e80f96372` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:11.565026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e64285d36bd45f288981e5e80f96372)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:11.569587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:11.619670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 852/1705] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:11 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:11.622445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d8f7a-6b7b-4c61-8a6a-9aeaf12c5cc3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:11.623634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d8f7a-6b7b-4c61-8a6a-9aeaf12c5cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:11.628617 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d8f7a-6b7b-4c61-8a6a-9aeaf12c5cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:11.663162 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-dd5d8f7a-6b7b-4c61-8a6a-9aeaf12c5cc3 None None] MFA Rules not processed for user `f2d1f86034b94e289bb1fc754f02f4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:11.664430 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dd5d8f7a-6b7b-4c61-8a6a-9aeaf12c5cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:11.761847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e64285d36bd45f288981e5e80f96372', 'request_id': 'req-9644d806-dea4-4cb3-9018-374654d616b0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:11.786342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 853/1706] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:11.798679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d8f7a-6b7b-4c61-8a6a-9aeaf12c5cc3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:11.798679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dd5d8f7a-6b7b-4c61-8a6a-9aeaf12c5cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:11.815847 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d8f7a-6b7b-4c61-8a6a-9aeaf12c5cc3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:11.874874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e64285d36bd45f288981e5e80f96372', 'request_id': 'req-9644d806-dea4-4cb3-9018-374654d616b0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:11.886930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e64285d36bd45f288981e5e80f96372', 'request_id': 'req-9644d806-dea4-4cb3-9018-374654d616b0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:11.922579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] The token cache is being invalidate because project 5e64285d36bd45f288981e5e80f96372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:11.922914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:11.928404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9644d806-dea4-4cb3-9018-374654d616b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:11.930716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 854/1707] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:11 2022] DELETE /identity/v3/projects/5e64285d36bd45f288981e5e80f96372 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:11.932066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf60eb-2130-4613-8666-c63673b72073 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:11.933052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf60eb-2130-4613-8666-c63673b72073 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:11.933487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf60eb-2130-4613-8666-c63673b72073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:11.975916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907675d-327c-40f5-978f-ba41c795da56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:11.976987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907675d-327c-40f5-978f-ba41c795da56 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:11.980185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907675d-327c-40f5-978f-ba41c795da56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:11.981565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907675d-327c-40f5-978f-ba41c795da56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:11.982520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907675d-327c-40f5-978f-ba41c795da56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:11.984993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2907675d-327c-40f5-978f-ba41c795da56 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:12.002331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-acaf60eb-2130-4613-8666-c63673b72073 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:12.014933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-acaf60eb-2130-4613-8666-c63673b72073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:12.014933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:12.022491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 855/1708] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:12.023559 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf60eb-2130-4613-8666-c63673b72073 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:12.025057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-acaf60eb-2130-4613-8666-c63673b72073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:12.040176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf60eb-2130-4613-8666-c63673b72073 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:12.047225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def49b7d-3bfa-402b-8de9-cbdfc59db333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:12.048138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def49b7d-3bfa-402b-8de9-cbdfc59db333 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:12.048414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def49b7d-3bfa-402b-8de9-cbdfc59db333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:12.049100 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def49b7d-3bfa-402b-8de9-cbdfc59db333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:12.049850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def49b7d-3bfa-402b-8de9-cbdfc59db333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:12.051499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def49b7d-3bfa-402b-8de9-cbdfc59db333 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:12.064899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 856/1709] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:12.065788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def49b7d-3bfa-402b-8de9-cbdfc59db333 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:12.071692 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-def49b7d-3bfa-402b-8de9-cbdfc59db333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:12.106280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 854/1710] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:11 2022] POST /identity/v3/projects => generated 379 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:12.106280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907675d-327c-40f5-978f-ba41c795da56 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:12.110483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2907675d-327c-40f5-978f-ba41c795da56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:12.213216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2907675d-327c-40f5-978f-ba41c795da56 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:12.241528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:12.246382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:12.246717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:12.247005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] PATH_INFO: `/v3/users/7060f13071764eb19fe4858f3ee68011` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:12.248485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7060f13071764eb19fe4858f3ee68011)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:12.250788 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:12.254593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def49b7d-3bfa-402b-8de9-cbdfc59db333 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:12.365257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba827-96b6-410a-a5cf-d6142e898832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:12.366183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba827-96b6-410a-a5cf-d6142e898832 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:12.366584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba827-96b6-410a-a5cf-d6142e898832 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:12.366967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba827-96b6-410a-a5cf-d6142e898832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:12.369060 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8a3ba827-96b6-410a-a5cf-d6142e898832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:12.476935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7060f13071764eb19fe4858f3ee68011'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:12.490777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7060f13071764eb19fe4858f3ee68011'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:12.507016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ba827-96b6-410a-a5cf-d6142e898832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:12.508778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ba827-96b6-410a-a5cf-d6142e898832 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:12.519230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7060f13071764eb19fe4858f3ee68011'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:12.531254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 857/1711] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:12.532630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e6988-a4fa-4e95-b350-47fcf0a88174 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:12.532957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e6988-a4fa-4e95-b350-47fcf0a88174 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:12.533236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e6988-a4fa-4e95-b350-47fcf0a88174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:12.594335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7060f13071764eb19fe4858f3ee68011'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:12.595008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7060f13071764eb19fe4858f3ee68011'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:12.603407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 855/1712] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:12 2022] DELETE /identity/v3/users/7060f13071764eb19fe4858f3ee68011 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:12.603979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:12.634439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-152e6988-a4fa-4e95-b350-47fcf0a88174 None None] MFA Rules not processed for user `6890f2e62cfd46999791d4a10a81f95c`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:12.635695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-152e6988-a4fa-4e95-b350-47fcf0a88174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:12.641668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:12.691715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:12.863554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 858/1713] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:12.864573 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e6988-a4fa-4e95-b350-47fcf0a88174 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:12.871581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-152e6988-a4fa-4e95-b350-47fcf0a88174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:12.875031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb997967-ef7a-40ff-a94f-a519f82c01c6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:12.887685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e6988-a4fa-4e95-b350-47fcf0a88174 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:12.887961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a1970-b697-4c63-9c5f-1d9b56c13923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:12.888440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a1970-b697-4c63-9c5f-1d9b56c13923 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:12.888940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a1970-b697-4c63-9c5f-1d9b56c13923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:12.891218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a1970-b697-4c63-9c5f-1d9b56c13923 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:12.892352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a1970-b697-4c63-9c5f-1d9b56c13923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:12.892668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a1970-b697-4c63-9c5f-1d9b56c13923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:12.893438 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb61eb-94d7-403a-beb4-cccfa2076d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:12.894511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb61eb-94d7-403a-beb4-cccfa2076d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:12.894657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a1970-b697-4c63-9c5f-1d9b56c13923 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:12.894826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb61eb-94d7-403a-beb4-cccfa2076d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:12.895076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb61eb-94d7-403a-beb4-cccfa2076d60 admin admin] PATH_INFO: `/v3/projects/9d04e2c9f3ca43e98eff6a0c4703810e/users/ba9a6eb1ff9d4af0970503fe85982043/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:12.920039 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb61eb-94d7-403a-beb4-cccfa2076d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d04e2c9f3ca43e98eff6a0c4703810e, user_id=ba9a6eb1ff9d4af0970503fe85982043, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:12.924931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb61eb-94d7-403a-beb4-cccfa2076d60 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:12.942493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 856/1714] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:12.942493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a1970-b697-4c63-9c5f-1d9b56c13923 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:12.943198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:12.950110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4f4a1970-b697-4c63-9c5f-1d9b56c13923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:13.042263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 859/1715] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:12 2022] PUT /identity/v3/projects/9d04e2c9f3ca43e98eff6a0c4703810e/users/ba9a6eb1ff9d4af0970503fe85982043/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:13.044543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb61eb-94d7-403a-beb4-cccfa2076d60 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:13.061497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f6fb61eb-94d7-403a-beb4-cccfa2076d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:13.061715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a1970-b697-4c63-9c5f-1d9b56c13923 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:13.094568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:13.256065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883e01a-5a13-4e98-9db0-e0cb3d33cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:13.257192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883e01a-5a13-4e98-9db0-e0cb3d33cad8 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:13.257628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883e01a-5a13-4e98-9db0-e0cb3d33cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:13.258091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883e01a-5a13-4e98-9db0-e0cb3d33cad8 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:13.259699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883e01a-5a13-4e98-9db0-e0cb3d33cad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:13.266104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883e01a-5a13-4e98-9db0-e0cb3d33cad8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:13.336065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb61eb-94d7-403a-beb4-cccfa2076d60 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:13.343036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855cc04-fb91-4ce8-ad8d-94350d592c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:13.343771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855cc04-fb91-4ce8-ad8d-94350d592c64 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:13.344054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855cc04-fb91-4ce8-ad8d-94350d592c64 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:13.344374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855cc04-fb91-4ce8-ad8d-94350d592c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:13.419596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 857/1716] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:12 2022] POST /identity/v3/users => generated 296 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:13.420668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883e01a-5a13-4e98-9db0-e0cb3d33cad8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:13.444749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855cc04-fb91-4ce8-ad8d-94350d592c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:13.447729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883e01a-5a13-4e98-9db0-e0cb3d33cad8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:13.452981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855cc04-fb91-4ce8-ad8d-94350d592c64 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:13.468330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 860/1717] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:13.474228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac23e62-35db-4be6-be1b-4463716306e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:13.474941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac23e62-35db-4be6-be1b-4463716306e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:13.475231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac23e62-35db-4be6-be1b-4463716306e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:13.475524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac23e62-35db-4be6-be1b-4463716306e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:13.476909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac23e62-35db-4be6-be1b-4463716306e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:13.479109 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac23e62-35db-4be6-be1b-4463716306e3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:13.479833 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855cc04-fb91-4ce8-ad8d-94350d592c64 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:13.483737 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2855cc04-fb91-4ce8-ad8d-94350d592c64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:13.504349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:13.517186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855cc04-fb91-4ce8-ad8d-94350d592c64 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:13.523384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:13.524474 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:13.524764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:13.525153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] PATH_INFO: `/v3/users/d528ebf2e8b64a51bc92b9f636902da2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:13.526664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d528ebf2e8b64a51bc92b9f636902da2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:13.529210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:13.606265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 858/1718] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:13 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:13.606265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac23e62-35db-4be6-be1b-4463716306e3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:13.775019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac23e62-35db-4be6-be1b-4463716306e3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:13.781051 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0c1cf-8b52-4856-a123-04b072ed5ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:13.782208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0c1cf-8b52-4856-a123-04b072ed5ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:13.782830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0c1cf-8b52-4856-a123-04b072ed5ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:13.783311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0c1cf-8b52-4856-a123-04b072ed5ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:13.784283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0c1cf-8b52-4856-a123-04b072ed5ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:13.786321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0c1cf-8b52-4856-a123-04b072ed5ae0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:13.789816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd528ebf2e8b64a51bc92b9f636902da2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:13.812278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 859/1719] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:13.817974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b523e21-6c14-4b0c-864c-c714dc30c20e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:13.819852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd528ebf2e8b64a51bc92b9f636902da2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:13.820971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b523e21-6c14-4b0c-864c-c714dc30c20e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:13.821463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b523e21-6c14-4b0c-864c-c714dc30c20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:13.843448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd528ebf2e8b64a51bc92b9f636902da2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:13.904350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4b523e21-6c14-4b0c-864c-c714dc30c20e None None] MFA Rules not processed for user `baaf254c2b28444bab7f1c7cc3fbc42c`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:13.906414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4b523e21-6c14-4b0c-864c-c714dc30c20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:13.960945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd528ebf2e8b64a51bc92b9f636902da2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:13.962671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd528ebf2e8b64a51bc92b9f636902da2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:13.982952 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 861/1720] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:13 2022] DELETE /identity/v3/users/d528ebf2e8b64a51bc92b9f636902da2 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:13.984442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:13.986685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:14.067886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 860/1721] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:14.069312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac7056-c1b3-4113-a5fb-e5c0d8cf6e25 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:14.069687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac7056-c1b3-4113-a5fb-e5c0d8cf6e25 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:14.070118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac7056-c1b3-4113-a5fb-e5c0d8cf6e25 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:14.071728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 861/1722] 10.222.0.88 () {54 vars in 912 bytes} [Mon Jul 18 16:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:50:14.072489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac7056-c1b3-4113-a5fb-e5c0d8cf6e25 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:14.074448 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-23ac7056-c1b3-4113-a5fb-e5c0d8cf6e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:14.098008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9254486-8625-4a15-8846-fb69dbfaa7b0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:14.104241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ab869-35ab-4f9d-ad2d-30b23373aa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:14.105947 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ab869-35ab-4f9d-ad2d-30b23373aa72 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:14.105947 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ab869-35ab-4f9d-ad2d-30b23373aa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:14.106569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ab869-35ab-4f9d-ad2d-30b23373aa72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:14.107987 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ab869-35ab-4f9d-ad2d-30b23373aa72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:14.109963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ab869-35ab-4f9d-ad2d-30b23373aa72 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:14.123439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 862/1723] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:14.138750 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:14.145622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ab869-35ab-4f9d-ad2d-30b23373aa72 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:14.150201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d03ab869-35ab-4f9d-ad2d-30b23373aa72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:14.163180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac7056-c1b3-4113-a5fb-e5c0d8cf6e25 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:14.170429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cafbb-c6a4-4e51-8666-66cb9869a560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:14.171921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cafbb-c6a4-4e51-8666-66cb9869a560 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:14.172444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cafbb-c6a4-4e51-8666-66cb9869a560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:14.172548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ab869-35ab-4f9d-ad2d-30b23373aa72 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:14.173073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cafbb-c6a4-4e51-8666-66cb9869a560 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:14.174042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5cafbb-c6a4-4e51-8666-66cb9869a560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:14.176982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5cafbb-c6a4-4e51-8666-66cb9869a560 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:14.179567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b06439-e63f-42a1-95ce-13d6511c7e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:14.190193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b06439-e63f-42a1-95ce-13d6511c7e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:14.190651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b06439-e63f-42a1-95ce-13d6511c7e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:14.190952 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b06439-e63f-42a1-95ce-13d6511c7e8e admin admin] PATH_INFO: `/v3/projects/9d04e2c9f3ca43e98eff6a0c4703810e/users/ba9a6eb1ff9d4af0970503fe85982043/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:14.193741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b06439-e63f-42a1-95ce-13d6511c7e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d04e2c9f3ca43e98eff6a0c4703810e, user_id=ba9a6eb1ff9d4af0970503fe85982043, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:14.197066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b06439-e63f-42a1-95ce-13d6511c7e8e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:14.209708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 863/1724] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:14 2022] PUT /identity/v3/projects/9d04e2c9f3ca43e98eff6a0c4703810e/users/ba9a6eb1ff9d4af0970503fe85982043/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:14.223950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b06439-e63f-42a1-95ce-13d6511c7e8e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:14.225946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d4b06439-e63f-42a1-95ce-13d6511c7e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:14.261865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:14.288929 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 862/1725] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:14 2022] POST /identity/v3/users => generated 288 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:14.292141 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91025911-0efe-4951-bee7-00f25b90728a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:14.292567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91025911-0efe-4951-bee7-00f25b90728a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:14.292853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91025911-0efe-4951-bee7-00f25b90728a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:14.334564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b06439-e63f-42a1-95ce-13d6511c7e8e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:14.342662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:14.343769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:14.344174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:14.344460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] PATH_INFO: `/v3/projects/bd5f3b0f6d7944bbadb6c86a2000c13e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:14.345773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd5f3b0f6d7944bbadb6c86a2000c13e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:14.348373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:14.351759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-91025911-0efe-4951-bee7-00f25b90728a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:14.356016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-91025911-0efe-4951-bee7-00f25b90728a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:14.383750 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 863/1726] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:14.391020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb71b2-b696-4226-abe2-8a627ea64ec8 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:14.393856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb71b2-b696-4226-abe2-8a627ea64ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:14.393856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb71b2-b696-4226-abe2-8a627ea64ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:14.414503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-64cb71b2-b696-4226-abe2-8a627ea64ec8 None None] MFA Rules not processed for user `baaf254c2b28444bab7f1c7cc3fbc42c`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:14.416201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-64cb71b2-b696-4226-abe2-8a627ea64ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:14.528035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd5f3b0f6d7944bbadb6c86a2000c13e', 'request_id': 'req-f190db76-f504-4e1d-bed0-e075e1b7ea4e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:14.538725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 864/1727] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:14.545765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599f8c3-ecfd-4005-a322-353a2cb60410 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:14.547894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599f8c3-ecfd-4005-a322-353a2cb60410 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:14.547894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599f8c3-ecfd-4005-a322-353a2cb60410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:14.639210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7599f8c3-ecfd-4005-a322-353a2cb60410 None None] MFA Rules not processed for user `de8484c3b114467ea59bebb2460ed546`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:14.641098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7599f8c3-ecfd-4005-a322-353a2cb60410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:14.651641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd5f3b0f6d7944bbadb6c86a2000c13e', 'request_id': 'req-f190db76-f504-4e1d-bed0-e075e1b7ea4e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:14.665992 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd5f3b0f6d7944bbadb6c86a2000c13e', 'request_id': 'req-f190db76-f504-4e1d-bed0-e075e1b7ea4e'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:14.691490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] The token cache is being invalidate because project bd5f3b0f6d7944bbadb6c86a2000c13e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:14.692754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:14.693985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:14.697944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 864/1728] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:14 2022] DELETE /identity/v3/projects/bd5f3b0f6d7944bbadb6c86a2000c13e => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:14.699010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:14.714986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:14.747087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 865/1729] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:50:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:14.748558 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b73a2a-e25f-4f80-a03f-fa5f7202139b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:14.748696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b73a2a-e25f-4f80-a03f-fa5f7202139b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:14.748974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b73a2a-e25f-4f80-a03f-fa5f7202139b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:14.812869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190db76-f504-4e1d-bed0-e075e1b7ea4e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:14.818572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-45b73a2a-e25f-4f80-a03f-fa5f7202139b None None] MFA Rules not processed for user `ba9a6eb1ff9d4af0970503fe85982043`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:14.820156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-45b73a2a-e25f-4f80-a03f-fa5f7202139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:14.824061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee376ec3-342d-483b-b14a-555c23bc375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:14.825239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee376ec3-342d-483b-b14a-555c23bc375d admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:14.833618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee376ec3-342d-483b-b14a-555c23bc375d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:14.833618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee376ec3-342d-483b-b14a-555c23bc375d admin admin] PATH_INFO: `/v3/projects/37e0a6554bf04c6eaa4008a4532f299f/users/d71453803e284c929ceb65456809db13/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:14.853554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee376ec3-342d-483b-b14a-555c23bc375d admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e0a6554bf04c6eaa4008a4532f299f, user_id=d71453803e284c929ceb65456809db13, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:14.857424 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee376ec3-342d-483b-b14a-555c23bc375d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:14.963714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 865/1730] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:14 2022] PUT /identity/v3/projects/37e0a6554bf04c6eaa4008a4532f299f/users/d71453803e284c929ceb65456809db13/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:14.964911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee376ec3-342d-483b-b14a-555c23bc375d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:14.968070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ee376ec3-342d-483b-b14a-555c23bc375d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:15.009413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 866/1731] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:15.010382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b73a2a-e25f-4f80-a03f-fa5f7202139b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:15.014866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-45b73a2a-e25f-4f80-a03f-fa5f7202139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:15.048123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b73a2a-e25f-4f80-a03f-fa5f7202139b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:15.126354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee376ec3-342d-483b-b14a-555c23bc375d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:15.139432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567a4d3-7ab8-4f8f-a7ad-78c774668b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:15.139432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a4d3-7ab8-4f8f-a7ad-78c774668b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:15.139432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a4d3-7ab8-4f8f-a7ad-78c774668b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:15.139432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a4d3-7ab8-4f8f-a7ad-78c774668b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:15.140637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567a4d3-7ab8-4f8f-a7ad-78c774668b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:15.144428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567a4d3-7ab8-4f8f-a7ad-78c774668b4d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:15.174285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 866/1732] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:15.175717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567a4d3-7ab8-4f8f-a7ad-78c774668b4d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:15.178403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2567a4d3-7ab8-4f8f-a7ad-78c774668b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:15.216186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c77b56-2578-4370-9a1a-fa2a178472e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:15.216865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c77b56-2578-4370-9a1a-fa2a178472e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:15.217180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c77b56-2578-4370-9a1a-fa2a178472e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:15.217460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c77b56-2578-4370-9a1a-fa2a178472e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:15.225526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d2c77b56-2578-4370-9a1a-fa2a178472e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:15.250630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c77b56-2578-4370-9a1a-fa2a178472e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:15.252751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c77b56-2578-4370-9a1a-fa2a178472e6 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:15.263133 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:15.274173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 867/1733] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:15.280983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65672cd-3ffb-4149-9744-71c5d96d4eef None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:15.281338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65672cd-3ffb-4149-9744-71c5d96d4eef None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:15.281626 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65672cd-3ffb-4149-9744-71c5d96d4eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:15.323090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e65672cd-3ffb-4149-9744-71c5d96d4eef None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:15.328256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e65672cd-3ffb-4149-9744-71c5d96d4eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:15.331542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567a4d3-7ab8-4f8f-a7ad-78c774668b4d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:15.352594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 868/1734] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:15.352594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65672cd-3ffb-4149-9744-71c5d96d4eef None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:15.358155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699561d-2392-4664-a397-5067f09d437f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:15.361088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699561d-2392-4664-a397-5067f09d437f service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:15.364852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699561d-2392-4664-a397-5067f09d437f service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:15.365258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699561d-2392-4664-a397-5067f09d437f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:15.371299 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1699561d-2392-4664-a397-5067f09d437f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:15.385191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65672cd-3ffb-4149-9744-71c5d96d4eef None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:15.400756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865a8a5-d445-454d-b236-b356ab65dcaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:15.401790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865a8a5-d445-454d-b236-b356ab65dcaa service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:15.402516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865a8a5-d445-454d-b236-b356ab65dcaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:15.402980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865a8a5-d445-454d-b236-b356ab65dcaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:15.429585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865a8a5-d445-454d-b236-b356ab65dcaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:15.432002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865a8a5-d445-454d-b236-b356ab65dcaa service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:15.450688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 869/1735] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:15.451770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865a8a5-d445-454d-b236-b356ab65dcaa service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:15.466778 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865a8a5-d445-454d-b236-b356ab65dcaa service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:15.475195 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d633b70-c63a-463e-baf5-ddb9bd2dcc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:15.476218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d633b70-c63a-463e-baf5-ddb9bd2dcc84 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:15.478467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d633b70-c63a-463e-baf5-ddb9bd2dcc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:15.478467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d633b70-c63a-463e-baf5-ddb9bd2dcc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:15.478467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d633b70-c63a-463e-baf5-ddb9bd2dcc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:15.480777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d633b70-c63a-463e-baf5-ddb9bd2dcc84 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:15.503797 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 870/1736] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:15.509646 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d633b70-c63a-463e-baf5-ddb9bd2dcc84 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:15.519515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9d633b70-c63a-463e-baf5-ddb9bd2dcc84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:15.581105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699561d-2392-4664-a397-5067f09d437f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:15.583311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699561d-2392-4664-a397-5067f09d437f service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:15.607922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:15.618113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 867/1737] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:15.618430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699561d-2392-4664-a397-5067f09d437f service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:15.650128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699561d-2392-4664-a397-5067f09d437f service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:15.665650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6606e-d944-4993-9d07-f1e8a98832dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:15.666748 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6606e-d944-4993-9d07-f1e8a98832dd admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:15.667054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6606e-d944-4993-9d07-f1e8a98832dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:15.667540 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6606e-d944-4993-9d07-f1e8a98832dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:15.669365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6606e-d944-4993-9d07-f1e8a98832dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:15.675869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6606e-d944-4993-9d07-f1e8a98832dd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:15.677679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d633b70-c63a-463e-baf5-ddb9bd2dcc84 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:15.686047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27215acb-d2d9-40ee-9806-4a5f1ca91e4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:15.687143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27215acb-d2d9-40ee-9806-4a5f1ca91e4c service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:15.688564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27215acb-d2d9-40ee-9806-4a5f1ca91e4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:15.689232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27215acb-d2d9-40ee-9806-4a5f1ca91e4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:15.708929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 868/1738] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:15.708929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318b196-bbff-413f-8c9d-6432aabecaff None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:15.708929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318b196-bbff-413f-8c9d-6432aabecaff None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:15.708929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318b196-bbff-413f-8c9d-6432aabecaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:15.743887 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9318b196-bbff-413f-8c9d-6432aabecaff None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:15.744521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9318b196-bbff-413f-8c9d-6432aabecaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:15.768297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:15.769235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 869/1739] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:15.771256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318b196-bbff-413f-8c9d-6432aabecaff None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:15.787137 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318b196-bbff-413f-8c9d-6432aabecaff None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:15.790249 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:15.795862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22550677-6239-4a18-b1af-6f9cba31f42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:15.796545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22550677-6239-4a18-b1af-6f9cba31f42f admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:15.796844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22550677-6239-4a18-b1af-6f9cba31f42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:15.797191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22550677-6239-4a18-b1af-6f9cba31f42f admin admin] PATH_INFO: `/v3/projects/37e0a6554bf04c6eaa4008a4532f299f/users/d71453803e284c929ceb65456809db13/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:15.799280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22550677-6239-4a18-b1af-6f9cba31f42f admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e0a6554bf04c6eaa4008a4532f299f, user_id=d71453803e284c929ceb65456809db13, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:15.803256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22550677-6239-4a18-b1af-6f9cba31f42f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:15.824623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 870/1740] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:15 2022] PUT /identity/v3/projects/37e0a6554bf04c6eaa4008a4532f299f/users/d71453803e284c929ceb65456809db13/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:15.825597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22550677-6239-4a18-b1af-6f9cba31f42f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:15.870010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27215acb-d2d9-40ee-9806-4a5f1ca91e4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:15.872150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27215acb-d2d9-40ee-9806-4a5f1ca91e4c service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:15.922391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 871/1741] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:15 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:15.924375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27215acb-d2d9-40ee-9806-4a5f1ca91e4c service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:15.964852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27215acb-d2d9-40ee-9806-4a5f1ca91e4c service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:16.014945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22550677-6239-4a18-b1af-6f9cba31f42f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:16.029133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3ddf-03d2-4273-8477-41bb59a81fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:16.030051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3ddf-03d2-4273-8477-41bb59a81fff admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:16.030337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3ddf-03d2-4273-8477-41bb59a81fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:16.031036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3ddf-03d2-4273-8477-41bb59a81fff admin admin] PATH_INFO: `/v3/projects/c445168cdf3f447eb9072e3f6da253d4/users/30c0536cbbd94332a3e5a1eaf4405e0d/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:16.070103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f3ddf-03d2-4273-8477-41bb59a81fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c445168cdf3f447eb9072e3f6da253d4, user_id=30c0536cbbd94332a3e5a1eaf4405e0d, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:16.074584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f3ddf-03d2-4273-8477-41bb59a81fff admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:16.130562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ddf1c-b24e-4a6a-b08f-47546e62743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:16.131649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ddf1c-b24e-4a6a-b08f-47546e62743d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:16.137172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ddf1c-b24e-4a6a-b08f-47546e62743d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:16.137455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ddf1c-b24e-4a6a-b08f-47546e62743d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:16.138218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ddf1c-b24e-4a6a-b08f-47546e62743d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:16.138585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ddf1c-b24e-4a6a-b08f-47546e62743d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:16.146045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ddf1c-b24e-4a6a-b08f-47546e62743d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:16.154907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:16.190526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 872/1742] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:16.190526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f171ccb-aece-469e-beef-baf31ed039fd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:16.190526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f171ccb-aece-469e-beef-baf31ed039fd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:16.190526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f171ccb-aece-469e-beef-baf31ed039fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:16.237791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 871/1743] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:15 2022] PUT /identity/v3/projects/c445168cdf3f447eb9072e3f6da253d4/users/30c0536cbbd94332a3e5a1eaf4405e0d/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:16.240810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3ddf-03d2-4273-8477-41bb59a81fff admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:16.243943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4f171ccb-aece-469e-beef-baf31ed039fd None None] MFA Rules not processed for user `d71453803e284c929ceb65456809db13`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:16.245484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4f171ccb-aece-469e-beef-baf31ed039fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:16.384708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 873/1744] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:16.384708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa1794-1e51-45bb-9c14-62509722219e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:16.384708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa1794-1e51-45bb-9c14-62509722219e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:16.384708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa1794-1e51-45bb-9c14-62509722219e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:16.394582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:16.430453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:16.488523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5dfa1794-1e51-45bb-9c14-62509722219e None None] MFA Rules not processed for user `30c0536cbbd94332a3e5a1eaf4405e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:16.496054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5dfa1794-1e51-45bb-9c14-62509722219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:16.544466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3ddf-03d2-4273-8477-41bb59a81fff admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:16.556411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a5019-0c7c-4e84-8516-64f33ca7d6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:16.556748 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a5019-0c7c-4e84-8516-64f33ca7d6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:16.557129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a5019-0c7c-4e84-8516-64f33ca7d6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:16.557473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a5019-0c7c-4e84-8516-64f33ca7d6f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:16.558186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a5019-0c7c-4e84-8516-64f33ca7d6f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:16.563996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a5019-0c7c-4e84-8516-64f33ca7d6f9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:16.596311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:16.689545 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 874/1745] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:16.694156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa1794-1e51-45bb-9c14-62509722219e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:16.694156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5dfa1794-1e51-45bb-9c14-62509722219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:16.705465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 872/1746] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:16 2022] POST /identity/v3/projects => generated 387 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:16.706181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14af36d-8f59-4499-bd49-866bb10e9dad None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:16.706645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14af36d-8f59-4499-bd49-866bb10e9dad None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:16.707079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14af36d-8f59-4499-bd49-866bb10e9dad None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:16.721338 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 873/1747] 10.222.0.88 () {54 vars in 912 bytes} [Mon Jul 18 16:50:16 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:50:16.722280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14af36d-8f59-4499-bd49-866bb10e9dad None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:16.864991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa1794-1e51-45bb-9c14-62509722219e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:16.874855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:16.903936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14af36d-8f59-4499-bd49-866bb10e9dad None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:16.909380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1af5e-8028-40af-87d1-a03ab1da485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:16.910723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1af5e-8028-40af-87d1-a03ab1da485e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:16.911061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1af5e-8028-40af-87d1-a03ab1da485e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:16.911345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e1af5e-8028-40af-87d1-a03ab1da485e admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:16.912060 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1af5e-8028-40af-87d1-a03ab1da485e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:16.915258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e1af5e-8028-40af-87d1-a03ab1da485e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:16.966355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00ad07-8a4f-4852-b20f-2452bc5b41e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:16.967274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00ad07-8a4f-4852-b20f-2452bc5b41e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:16.967628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00ad07-8a4f-4852-b20f-2452bc5b41e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:16.967969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00ad07-8a4f-4852-b20f-2452bc5b41e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:17.060695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00ad07-8a4f-4852-b20f-2452bc5b41e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:17.062417 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00ad07-8a4f-4852-b20f-2452bc5b41e5 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:17.078444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 874/1748] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:16 2022] POST /identity/v3/projects => generated 375 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:17.081726 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1af5e-8028-40af-87d1-a03ab1da485e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:17.092586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 875/1749] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:17.094526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11e754-650a-4077-8ce9-0f783d44211a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:17.095406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11e754-650a-4077-8ce9-0f783d44211a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:17.095964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11e754-650a-4077-8ce9-0f783d44211a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:17.111583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:17.146174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9c11e754-650a-4077-8ce9-0f783d44211a None None] MFA Rules not processed for user `de8484c3b114467ea59bebb2460ed546`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:17.149020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9c11e754-650a-4077-8ce9-0f783d44211a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:17.333403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e1af5e-8028-40af-87d1-a03ab1da485e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:17.343077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce82bf-66e1-4c7d-9e57-b3613bed8ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:17.344331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce82bf-66e1-4c7d-9e57-b3613bed8ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:17.344971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce82bf-66e1-4c7d-9e57-b3613bed8ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:17.345575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce82bf-66e1-4c7d-9e57-b3613bed8ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:17.346563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce82bf-66e1-4c7d-9e57-b3613bed8ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:17.349164 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce82bf-66e1-4c7d-9e57-b3613bed8ee9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:17.356365 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 876/1750] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:50:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:17.357485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11e754-650a-4077-8ce9-0f783d44211a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:17.360869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9c11e754-650a-4077-8ce9-0f783d44211a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:17.381641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11e754-650a-4077-8ce9-0f783d44211a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:17.387618 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:17.388909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:17.389308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:17.389865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] PATH_INFO: `/v3/projects/7a7f46101a294535bfc2a13a23e7a884` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:17.391918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7f46101a294535bfc2a13a23e7a884)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:17.394379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:17.495502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 875/1751] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:17 2022] POST /identity/v3/users => generated 294 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:17.496577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce82bf-66e1-4c7d-9e57-b3613bed8ee9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:17.510191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce82bf-66e1-4c7d-9e57-b3613bed8ee9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:17.517470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f2b14b-b2c7-4e11-9481-b6ef3180187a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:17.518925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2b14b-b2c7-4e11-9481-b6ef3180187a admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:17.519225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2b14b-b2c7-4e11-9481-b6ef3180187a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:17.519508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2b14b-b2c7-4e11-9481-b6ef3180187a admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:17.520977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2b14b-b2c7-4e11-9481-b6ef3180187a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:17.523034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f2b14b-b2c7-4e11-9481-b6ef3180187a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:17.631143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7f46101a294535bfc2a13a23e7a884', 'request_id': 'req-4991c968-fa5b-4ff3-b796-9db95f3bc855'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:17.643058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7f46101a294535bfc2a13a23e7a884', 'request_id': 'req-4991c968-fa5b-4ff3-b796-9db95f3bc855'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:17.652894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7f46101a294535bfc2a13a23e7a884', 'request_id': 'req-4991c968-fa5b-4ff3-b796-9db95f3bc855'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:17.663484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 876/1752] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:17 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:17.665126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c87486-891a-4333-b523-8e4c48e99660 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:17.665458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c87486-891a-4333-b523-8e4c48e99660 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:17.665835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c87486-891a-4333-b523-8e4c48e99660 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:17.667469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 877/1753] 10.222.0.88 () {54 vars in 961 bytes} [Mon Jul 18 16:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:50:17.668297 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c87486-891a-4333-b523-8e4c48e99660 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:17.670050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-16c87486-891a-4333-b523-8e4c48e99660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:17.743357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c87486-891a-4333-b523-8e4c48e99660 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:17.748606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb732494-2bc5-44e3-b5e0-f81e0aa3c32d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:17.749783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb732494-2bc5-44e3-b5e0-f81e0aa3c32d service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:17.749783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb732494-2bc5-44e3-b5e0-f81e0aa3c32d service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:17.749929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb732494-2bc5-44e3-b5e0-f81e0aa3c32d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:17.767238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb732494-2bc5-44e3-b5e0-f81e0aa3c32d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:17.767238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb732494-2bc5-44e3-b5e0-f81e0aa3c32d service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:17.771467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 878/1754] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:17.772830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ee5cd-3e53-4f7f-9a85-d6bf52048f1d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:17.773148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ee5cd-3e53-4f7f-9a85-d6bf52048f1d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:17.773425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ee5cd-3e53-4f7f-9a85-d6bf52048f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:17.792325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0a9ee5cd-3e53-4f7f-9a85-d6bf52048f1d None None] MFA Rules not processed for user `f2d1f86034b94e289bb1fc754f02f4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:17.793475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0a9ee5cd-3e53-4f7f-9a85-d6bf52048f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:17.801051 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] The token cache is being invalidate because project 7a7f46101a294535bfc2a13a23e7a884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:17.801619 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:17.807367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:17.807367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 877/1755] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:17 2022] DELETE /identity/v3/projects/7a7f46101a294535bfc2a13a23e7a884 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:17.807367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:17.809111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:17.886725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 879/1756] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:17.892776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991c968-fa5b-4ff3-b796-9db95f3bc855 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:17.897728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d0343-66e7-47ef-89b8-3b733feae727 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:17.899044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d0343-66e7-47ef-89b8-3b733feae727 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:17.899044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d0343-66e7-47ef-89b8-3b733feae727 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:17.899198 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d0343-66e7-47ef-89b8-3b733feae727 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:17.901145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6a1d0343-66e7-47ef-89b8-3b733feae727 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:17.904503 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ee5cd-3e53-4f7f-9a85-d6bf52048f1d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:17.909080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0a9ee5cd-3e53-4f7f-9a85-d6bf52048f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:17.920563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:17.967698 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:18.016721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d0343-66e7-47ef-89b8-3b733feae727 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:18.018378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d0343-66e7-47ef-89b8-3b733feae727 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:18.041501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 878/1757] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:18.042812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d0343-66e7-47ef-89b8-3b733feae727 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:18.049456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6a1d0343-66e7-47ef-89b8-3b733feae727 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:18.080443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ee5cd-3e53-4f7f-9a85-d6bf52048f1d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:18.101893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad828d-961b-4e52-9d31-606e4c152b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:18.102957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad828d-961b-4e52-9d31-606e4c152b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:18.103393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad828d-961b-4e52-9d31-606e4c152b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:18.105027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad828d-961b-4e52-9d31-606e4c152b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:18.106076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad828d-961b-4e52-9d31-606e4c152b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:18.109367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad828d-961b-4e52-9d31-606e4c152b6e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:18.131055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 880/1758] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:18.132391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad828d-961b-4e52-9d31-606e4c152b6e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:18.141073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-02ad828d-961b-4e52-9d31-606e4c152b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:18.218279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d0343-66e7-47ef-89b8-3b733feae727 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:18.221724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad828d-961b-4e52-9d31-606e4c152b6e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:18.229499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c74171-292d-4923-a9eb-42df15a6a3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:18.229869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67addc32-7d4f-4419-94b9-b43962aff262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:18.230202 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c74171-292d-4923-a9eb-42df15a6a3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:18.230494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c74171-292d-4923-a9eb-42df15a6a3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:18.230764 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67addc32-7d4f-4419-94b9-b43962aff262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:18.230865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c74171-292d-4923-a9eb-42df15a6a3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:18.231673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c74171-292d-4923-a9eb-42df15a6a3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:18.233050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67addc32-7d4f-4419-94b9-b43962aff262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:18.233351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67addc32-7d4f-4419-94b9-b43962aff262 admin admin] PATH_INFO: `/v3/users/8b2183a1b4c6417d9e0ceb56ddfad9a3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:18.233453 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c74171-292d-4923-a9eb-42df15a6a3c4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:18.239364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67addc32-7d4f-4419-94b9-b43962aff262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2183a1b4c6417d9e0ceb56ddfad9a3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:18.241593 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67addc32-7d4f-4419-94b9-b43962aff262 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:18.258197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:18.258694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 881/1759] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:18.261157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb233e-d7a1-4ddd-8fdf-950950b07b91 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:18.262240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb233e-d7a1-4ddd-8fdf-950950b07b91 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:18.263373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb233e-d7a1-4ddd-8fdf-950950b07b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:18.287962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1ebb233e-d7a1-4ddd-8fdf-950950b07b91 None None] MFA Rules not processed for user `22bd4dab1fcd4ad4a8be886646d89242`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:18.289618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1ebb233e-d7a1-4ddd-8fdf-950950b07b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:18.430129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 882/1760] 10.222.0.88 () {58 vars in 1058 bytes} [Mon Jul 18 16:50:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:18.431646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f05db-5661-4a04-9305-896aa8b41caa None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:18.431991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f05db-5661-4a04-9305-896aa8b41caa None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:18.432437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f05db-5661-4a04-9305-896aa8b41caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:18.456173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-859f05db-5661-4a04-9305-896aa8b41caa None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:18.458229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-859f05db-5661-4a04-9305-896aa8b41caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:18.484519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 883/1761] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:18.485446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f05db-5661-4a04-9305-896aa8b41caa None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:18.505138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f05db-5661-4a04-9305-896aa8b41caa None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:18.513169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24db579-ccff-431c-96fe-2a7780de5626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:18.513871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24db579-ccff-431c-96fe-2a7780de5626 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:18.514162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24db579-ccff-431c-96fe-2a7780de5626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:18.517395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24db579-ccff-431c-96fe-2a7780de5626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:18.518352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24db579-ccff-431c-96fe-2a7780de5626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:18.520132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24db579-ccff-431c-96fe-2a7780de5626 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:18.537166 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 884/1762] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:18.538234 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24db579-ccff-431c-96fe-2a7780de5626 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:18.553489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24db579-ccff-431c-96fe-2a7780de5626 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:18.559253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08227a3d-d0d4-4c78-bd75-0d712373eef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:18.560027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-67addc32-7d4f-4419-94b9-b43962aff262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2183a1b4c6417d9e0ceb56ddfad9a3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:18.560510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08227a3d-d0d4-4c78-bd75-0d712373eef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:18.560510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08227a3d-d0d4-4c78-bd75-0d712373eef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:18.560752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08227a3d-d0d4-4c78-bd75-0d712373eef0 admin admin] PATH_INFO: `/v3/projects/4c86610a0dd14c2c986be65ad74ea79f/users/e1d118a5d4e6487f90744ecf58271823/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:18.591728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-67addc32-7d4f-4419-94b9-b43962aff262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2183a1b4c6417d9e0ceb56ddfad9a3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:18.594002 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08227a3d-d0d4-4c78-bd75-0d712373eef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c86610a0dd14c2c986be65ad74ea79f, user_id=e1d118a5d4e6487f90744ecf58271823, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:18.601139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08227a3d-d0d4-4c78-bd75-0d712373eef0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:18.656361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-67addc32-7d4f-4419-94b9-b43962aff262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2183a1b4c6417d9e0ceb56ddfad9a3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:18.710912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 885/1763] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:18 2022] PUT /identity/v3/projects/4c86610a0dd14c2c986be65ad74ea79f/users/e1d118a5d4e6487f90744ecf58271823/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:18.711794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08227a3d-d0d4-4c78-bd75-0d712373eef0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:18.733321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-67addc32-7d4f-4419-94b9-b43962aff262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2183a1b4c6417d9e0ceb56ddfad9a3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:18.734436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-67addc32-7d4f-4419-94b9-b43962aff262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2183a1b4c6417d9e0ceb56ddfad9a3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:18.756091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 879/1764] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:18 2022] DELETE /identity/v3/users/8b2183a1b4c6417d9e0ceb56ddfad9a3 => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:18.767415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bebfb8-1368-4e6a-9581-ed59adb7b28f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:18.767415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bebfb8-1368-4e6a-9581-ed59adb7b28f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:18.769599 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bebfb8-1368-4e6a-9581-ed59adb7b28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:18.842166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-15bebfb8-1368-4e6a-9581-ed59adb7b28f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:18.857707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-15bebfb8-1368-4e6a-9581-ed59adb7b28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:18.912943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08227a3d-d0d4-4c78-bd75-0d712373eef0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:19.070476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24f788-eb60-4ac0-a5e7-8c8e57073cd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:19.071375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24f788-eb60-4ac0-a5e7-8c8e57073cd7 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:19.071660 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24f788-eb60-4ac0-a5e7-8c8e57073cd7 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:19.071943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24f788-eb60-4ac0-a5e7-8c8e57073cd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:19.078499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5b24f788-eb60-4ac0-a5e7-8c8e57073cd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:19.078676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 880/1765] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:19.078676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bebfb8-1368-4e6a-9581-ed59adb7b28f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:19.085352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-15bebfb8-1368-4e6a-9581-ed59adb7b28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:19.102338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bebfb8-1368-4e6a-9581-ed59adb7b28f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:19.103740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24f788-eb60-4ac0-a5e7-8c8e57073cd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:19.107657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ac3f9-e080-4413-a6d0-af982c07a642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:19.108870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ac3f9-e080-4413-a6d0-af982c07a642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:19.109263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ac3f9-e080-4413-a6d0-af982c07a642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:19.109614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ac3f9-e080-4413-a6d0-af982c07a642 admin admin] PATH_INFO: `/v3/projects/974112776ff744c98676805c8616cc48/users/58bfff564cd04ccdbac2f3c9d9d294af/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:19.121754 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:19.135220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ac3f9-e080-4413-a6d0-af982c07a642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974112776ff744c98676805c8616cc48, user_id=58bfff564cd04ccdbac2f3c9d9d294af, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:19.135427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24f788-eb60-4ac0-a5e7-8c8e57073cd7 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:19.138262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ac3f9-e080-4413-a6d0-af982c07a642 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:19.169924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 886/1766] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:19.169924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24f788-eb60-4ac0-a5e7-8c8e57073cd7 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:19.177372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5b24f788-eb60-4ac0-a5e7-8c8e57073cd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:19.182744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:19.210282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24f788-eb60-4ac0-a5e7-8c8e57073cd7 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:19.218508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 881/1767] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:19 2022] PUT /identity/v3/projects/974112776ff744c98676805c8616cc48/users/58bfff564cd04ccdbac2f3c9d9d294af/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:19.222071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c03058-2560-47ff-bd1a-601ec6427160 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:19.224904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c03058-2560-47ff-bd1a-601ec6427160 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:19.224904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c03058-2560-47ff-bd1a-601ec6427160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:19.314001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-42c03058-2560-47ff-bd1a-601ec6427160 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:19.314001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-42c03058-2560-47ff-bd1a-601ec6427160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:19.384935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273e80b-57a8-408a-a2fe-d31e7d250fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:19.385491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273e80b-57a8-408a-a2fe-d31e7d250fae admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:19.386310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273e80b-57a8-408a-a2fe-d31e7d250fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:19.386486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273e80b-57a8-408a-a2fe-d31e7d250fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:19.387655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273e80b-57a8-408a-a2fe-d31e7d250fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:19.394347 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273e80b-57a8-408a-a2fe-d31e7d250fae admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:19.453422 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 887/1768] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:19.464096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273e80b-57a8-408a-a2fe-d31e7d250fae admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:19.464096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d273e80b-57a8-408a-a2fe-d31e7d250fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:19.535195 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 882/1769] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:19.536661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c03058-2560-47ff-bd1a-601ec6427160 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:19.563453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c03058-2560-47ff-bd1a-601ec6427160 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:19.580264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4c1f0-28eb-4a5e-862b-0dcde1d3dddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:19.581652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4c1f0-28eb-4a5e-862b-0dcde1d3dddd admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:19.582341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4c1f0-28eb-4a5e-862b-0dcde1d3dddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:19.582728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4c1f0-28eb-4a5e-862b-0dcde1d3dddd admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:19.583569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4c1f0-28eb-4a5e-862b-0dcde1d3dddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:19.584022 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4c1f0-28eb-4a5e-862b-0dcde1d3dddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:19.585755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4c1f0-28eb-4a5e-862b-0dcde1d3dddd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:19.604302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 883/1770] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:19.604302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13f878-f79f-491d-868c-0a71309e879c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:19.604302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13f878-f79f-491d-868c-0a71309e879c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:19.604302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13f878-f79f-491d-868c-0a71309e879c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:19.651327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6c13f878-f79f-491d-868c-0a71309e879c None None] MFA Rules not processed for user `58bfff564cd04ccdbac2f3c9d9d294af`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:19.653929 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6c13f878-f79f-491d-868c-0a71309e879c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:19.669478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273e80b-57a8-408a-a2fe-d31e7d250fae admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:19.682609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8064b4-9daf-4f61-89dd-3b034ee996db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:19.683595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8064b4-9daf-4f61-89dd-3b034ee996db service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:19.683890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8064b4-9daf-4f61-89dd-3b034ee996db service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:19.684171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8064b4-9daf-4f61-89dd-3b034ee996db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:19.704636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8064b4-9daf-4f61-89dd-3b034ee996db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:19.706392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8064b4-9daf-4f61-89dd-3b034ee996db service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:19.737383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 888/1771] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:19.742328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8064b4-9daf-4f61-89dd-3b034ee996db service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:19.773691 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 884/1772] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:19.776331 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1bc09-eeec-4784-aa41-069936e01993 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:19.776692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1bc09-eeec-4784-aa41-069936e01993 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:19.777000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1bc09-eeec-4784-aa41-069936e01993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:19.793194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8064b4-9daf-4f61-89dd-3b034ee996db service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:19.802906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-62e1bc09-eeec-4784-aa41-069936e01993 None None] MFA Rules not processed for user `6890f2e62cfd46999791d4a10a81f95c`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:19.802906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-62e1bc09-eeec-4784-aa41-069936e01993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:19.807365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5b763-3293-47a9-a9d0-ea9a81787d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:19.808289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5b763-3293-47a9-a9d0-ea9a81787d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:19.808578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5b763-3293-47a9-a9d0-ea9a81787d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:19.808862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5b763-3293-47a9-a9d0-ea9a81787d64 admin admin] PATH_INFO: `/v3/projects/4c86610a0dd14c2c986be65ad74ea79f/users/e1d118a5d4e6487f90744ecf58271823/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:19.815251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5b763-3293-47a9-a9d0-ea9a81787d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c86610a0dd14c2c986be65ad74ea79f, user_id=e1d118a5d4e6487f90744ecf58271823, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:19.821626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5b763-3293-47a9-a9d0-ea9a81787d64 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:19.840327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:19.848452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 889/1773] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:19 2022] PUT /identity/v3/projects/4c86610a0dd14c2c986be65ad74ea79f/users/e1d118a5d4e6487f90744ecf58271823/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:19.849961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5b763-3293-47a9-a9d0-ea9a81787d64 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:19.862671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9bf5b763-3293-47a9-a9d0-ea9a81787d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:19.991338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 885/1774] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:19.991338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1bc09-eeec-4784-aa41-069936e01993 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:20.020875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1bc09-eeec-4784-aa41-069936e01993 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:20.024260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5b763-3293-47a9-a9d0-ea9a81787d64 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:20.034796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a82b9f-766f-4559-8da1-585e53ee4c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:20.036803 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a82b9f-766f-4559-8da1-585e53ee4c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:20.038496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a82b9f-766f-4559-8da1-585e53ee4c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:20.038970 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a82b9f-766f-4559-8da1-585e53ee4c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:20.101526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:20.185029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a82b9f-766f-4559-8da1-585e53ee4c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:20.192049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a82b9f-766f-4559-8da1-585e53ee4c84 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:20.212477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 890/1775] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:20.213821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a82b9f-766f-4559-8da1-585e53ee4c84 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:20.242943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3afc55-09a9-4ae5-a11e-bb703cd5049a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:20.244427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3afc55-09a9-4ae5-a11e-bb703cd5049a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:20.245198 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3afc55-09a9-4ae5-a11e-bb703cd5049a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:20.245518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a82b9f-766f-4559-8da1-585e53ee4c84 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:20.246235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3afc55-09a9-4ae5-a11e-bb703cd5049a admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:20.247907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3afc55-09a9-4ae5-a11e-bb703cd5049a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:20.255790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3afc55-09a9-4ae5-a11e-bb703cd5049a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:20.261198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68268687-3805-493f-8f77-07fb79bb1e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:20.264056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68268687-3805-493f-8f77-07fb79bb1e43 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:20.264614 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68268687-3805-493f-8f77-07fb79bb1e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:20.266143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68268687-3805-493f-8f77-07fb79bb1e43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:20.268770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68268687-3805-493f-8f77-07fb79bb1e43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:20.271541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68268687-3805-493f-8f77-07fb79bb1e43 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:20.496789 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 886/1776] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:19 2022] POST /identity/v3/projects => generated 403 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:20.500715 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 891/1777] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:20 2022] POST /identity/v3/projects => generated 387 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:20.500862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b87ae-a642-48cc-a8d8-db802c5e6621 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:20.502160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68268687-3805-493f-8f77-07fb79bb1e43 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:20.505195 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b87ae-a642-48cc-a8d8-db802c5e6621 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:20.506444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b87ae-a642-48cc-a8d8-db802c5e6621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:20.573102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f08b87ae-a642-48cc-a8d8-db802c5e6621 None None] MFA Rules not processed for user `e1d118a5d4e6487f90744ecf58271823`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:20.574700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f08b87ae-a642-48cc-a8d8-db802c5e6621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:20.674879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68268687-3805-493f-8f77-07fb79bb1e43 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:20.679906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1d593-f91a-4e71-b028-6558018046cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:20.680571 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d593-f91a-4e71-b028-6558018046cd admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:20.680854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d593-f91a-4e71-b028-6558018046cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:20.681161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d593-f91a-4e71-b028-6558018046cd admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:20.681857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1d593-f91a-4e71-b028-6558018046cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:20.683860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1d593-f91a-4e71-b028-6558018046cd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:20.689504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 887/1778] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:20.690683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b87ae-a642-48cc-a8d8-db802c5e6621 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:20.714597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b87ae-a642-48cc-a8d8-db802c5e6621 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:20.722282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33eea4-9d9e-4ea6-bf31-458563434ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:20.722971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33eea4-9d9e-4ea6-bf31-458563434ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:20.723340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33eea4-9d9e-4ea6-bf31-458563434ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:20.723623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33eea4-9d9e-4ea6-bf31-458563434ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:20.724329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33eea4-9d9e-4ea6-bf31-458563434ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:20.726371 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33eea4-9d9e-4ea6-bf31-458563434ff4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:20.794351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 888/1779] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:20 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:20.794351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33eea4-9d9e-4ea6-bf31-458563434ff4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:20.794351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6b33eea4-9d9e-4ea6-bf31-458563434ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:20.797584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 892/1780] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:20 2022] POST /identity/v3/users => generated 302 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:20.798589 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1d593-f91a-4e71-b028-6558018046cd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:20.816489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1d593-f91a-4e71-b028-6558018046cd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:20.828818 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b27ebc-21e7-4284-a024-6a72a876ced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:20.829545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b27ebc-21e7-4284-a024-6a72a876ced4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:20.829834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b27ebc-21e7-4284-a024-6a72a876ced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:20.830121 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b27ebc-21e7-4284-a024-6a72a876ced4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:20.830879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b27ebc-21e7-4284-a024-6a72a876ced4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:20.833677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b27ebc-21e7-4284-a024-6a72a876ced4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:20.861172 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 893/1781] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:20.861957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b27ebc-21e7-4284-a024-6a72a876ced4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:20.875365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:20.902376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b27ebc-21e7-4284-a024-6a72a876ced4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:20.902376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdc9f2-2b9b-4824-9cf2-f2fcf6b04660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:20.902376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdc9f2-2b9b-4824-9cf2-f2fcf6b04660 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:20.902376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdc9f2-2b9b-4824-9cf2-f2fcf6b04660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:20.902833 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdc9f2-2b9b-4824-9cf2-f2fcf6b04660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:20.902833 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdc9f2-2b9b-4824-9cf2-f2fcf6b04660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:20.902833 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdc9f2-2b9b-4824-9cf2-f2fcf6b04660 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:20.924737 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 894/1782] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:20.928917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdc9f2-2b9b-4824-9cf2-f2fcf6b04660 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:20.937148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33eea4-9d9e-4ea6-bf31-458563434ff4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:20.948802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c18488-74bf-45e5-bb03-45d3562fd66f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:20.949392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18488-74bf-45e5-bb03-45d3562fd66f service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:20.952006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdc9f2-2b9b-4824-9cf2-f2fcf6b04660 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:20.953073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18488-74bf-45e5-bb03-45d3562fd66f service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:20.953351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18488-74bf-45e5-bb03-45d3562fd66f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:20.958388 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b0c18488-74bf-45e5-bb03-45d3562fd66f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:20.964111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6595-e061-42ed-b57b-eacd884f1f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:20.965061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6595-e061-42ed-b57b-eacd884f1f06 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:20.965502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6595-e061-42ed-b57b-eacd884f1f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:20.965946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6595-e061-42ed-b57b-eacd884f1f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:20.966960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf6595-e061-42ed-b57b-eacd884f1f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:20.968723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf6595-e061-42ed-b57b-eacd884f1f06 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:20.982548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:20.986238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 895/1783] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:20.988213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6595-e061-42ed-b57b-eacd884f1f06 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:21.000328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6595-e061-42ed-b57b-eacd884f1f06 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:21.012773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf402bd-5001-4d7e-bafa-44d11160c377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:21.013722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf402bd-5001-4d7e-bafa-44d11160c377 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:21.014009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf402bd-5001-4d7e-bafa-44d11160c377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:21.014935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf402bd-5001-4d7e-bafa-44d11160c377 admin admin] PATH_INFO: `/v3/projects/4a8072ad73f54e0292c916118ac4c78d/users/4f6975ed169f491a9f9c2add4d414b62/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:21.046630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf402bd-5001-4d7e-bafa-44d11160c377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8072ad73f54e0292c916118ac4c78d, user_id=4f6975ed169f491a9f9c2add4d414b62, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:21.051006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf402bd-5001-4d7e-bafa-44d11160c377 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:21.110142 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c18488-74bf-45e5-bb03-45d3562fd66f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:21.112367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c18488-74bf-45e5-bb03-45d3562fd66f service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:21.121668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 896/1784] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:20 2022] PUT /identity/v3/projects/4a8072ad73f54e0292c916118ac4c78d/users/4f6975ed169f491a9f9c2add4d414b62/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:21.127244 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf402bd-5001-4d7e-bafa-44d11160c377 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:21.143815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 889/1785] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:21.144856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c18488-74bf-45e5-bb03-45d3562fd66f service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:21.163235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c18488-74bf-45e5-bb03-45d3562fd66f service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:21.251282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf402bd-5001-4d7e-bafa-44d11160c377 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:21.267490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60468d1-4dea-4184-9e06-bef328ba2e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:21.268520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60468d1-4dea-4184-9e06-bef328ba2e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:21.268907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60468d1-4dea-4184-9e06-bef328ba2e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:21.269191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60468d1-4dea-4184-9e06-bef328ba2e93 admin admin] PATH_INFO: `/v3/projects/31f43a812a4548f0b11effb0ef1dec1e/users/f13537780e3a490db7f19451e7be9e43/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:21.304585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd8735-92fd-4284-9dd3-30b170b399f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:21.305579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8735-92fd-4284-9dd3-30b170b399f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:21.305945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8735-92fd-4284-9dd3-30b170b399f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:21.306290 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd8735-92fd-4284-9dd3-30b170b399f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:21.309268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd8735-92fd-4284-9dd3-30b170b399f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:21.309450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60468d1-4dea-4184-9e06-bef328ba2e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f43a812a4548f0b11effb0ef1dec1e, user_id=f13537780e3a490db7f19451e7be9e43, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:21.311202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd8735-92fd-4284-9dd3-30b170b399f2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:21.312291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60468d1-4dea-4184-9e06-bef328ba2e93 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:21.353578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 890/1786] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:21.354578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd8735-92fd-4284-9dd3-30b170b399f2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:21.360319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b9bd8735-92fd-4284-9dd3-30b170b399f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:21.378593 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd8735-92fd-4284-9dd3-30b170b399f2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:21.385466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:21.386746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:21.387013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:21.387335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] PATH_INFO: `/v3/projects/f3a4ff367f144d8186422c24642d8038` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:21.388905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a4ff367f144d8186422c24642d8038)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:21.394159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:21.429072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 897/1787] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:21 2022] PUT /identity/v3/projects/31f43a812a4548f0b11effb0ef1dec1e/users/f13537780e3a490db7f19451e7be9e43/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:21.429072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60468d1-4dea-4184-9e06-bef328ba2e93 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:21.598918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60468d1-4dea-4184-9e06-bef328ba2e93 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:21.617683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a4ff367f144d8186422c24642d8038', 'request_id': 'req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:21.637971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a4ff367f144d8186422c24642d8038', 'request_id': 'req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:21.649537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a4ff367f144d8186422c24642d8038', 'request_id': 'req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:21.700705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:21.719368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e590dcf-f0ea-43ee-8a66-bc3ba7efa629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:21.720534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e590dcf-f0ea-43ee-8a66-bc3ba7efa629 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:21.721031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e590dcf-f0ea-43ee-8a66-bc3ba7efa629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:21.721312 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e590dcf-f0ea-43ee-8a66-bc3ba7efa629 admin admin] PATH_INFO: `/v3/projects/4a8072ad73f54e0292c916118ac4c78d/users/4f6975ed169f491a9f9c2add4d414b62/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:21.725585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e590dcf-f0ea-43ee-8a66-bc3ba7efa629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8072ad73f54e0292c916118ac4c78d, user_id=4f6975ed169f491a9f9c2add4d414b62, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:21.728264 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e590dcf-f0ea-43ee-8a66-bc3ba7efa629 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:21.743165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 898/1788] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:21 2022] PUT /identity/v3/projects/4a8072ad73f54e0292c916118ac4c78d/users/4f6975ed169f491a9f9c2add4d414b62/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:21.744736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4292-cdae-4f89-9138-84cd5a9e31b0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:21.745062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4292-cdae-4f89-9138-84cd5a9e31b0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:21.745357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4292-cdae-4f89-9138-84cd5a9e31b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:21.768281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c83d4292-cdae-4f89-9138-84cd5a9e31b0 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:21.769731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c83d4292-cdae-4f89-9138-84cd5a9e31b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:21.795782 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] The token cache is being invalidate because project f3a4ff367f144d8186422c24642d8038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:21.796107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:21.798692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-eb5bc5d4-9930-4b95-921d-b7bebf81ee66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:21.801156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 891/1789] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:21 2022] DELETE /identity/v3/projects/f3a4ff367f144d8186422c24642d8038 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:21.807451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d8d7a-2533-48d3-a546-d8909c4e03b3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:21.807451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d8d7a-2533-48d3-a546-d8909c4e03b3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:21.807451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d8d7a-2533-48d3-a546-d8909c4e03b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:21.850183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e70d8d7a-2533-48d3-a546-d8909c4e03b3 None None] MFA Rules not processed for user `4f6975ed169f491a9f9c2add4d414b62`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:21.852414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e70d8d7a-2533-48d3-a546-d8909c4e03b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:21.899823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 899/1790] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:21.900708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:21.901458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d4292-cdae-4f89-9138-84cd5a9e31b0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:21.903010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c83d4292-cdae-4f89-9138-84cd5a9e31b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:21.918917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d4292-cdae-4f89-9138-84cd5a9e31b0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:21.939957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:21.942555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:21.943221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:21.943889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] PATH_INFO: `/v3/users/ba9a6eb1ff9d4af0970503fe85982043` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:21.946200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba9a6eb1ff9d4af0970503fe85982043)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:21.948710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:21.968932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 892/1791] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:21.970024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d8d7a-2533-48d3-a546-d8909c4e03b3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:21.971545 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e70d8d7a-2533-48d3-a546-d8909c4e03b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:22.035116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d8d7a-2533-48d3-a546-d8909c4e03b3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:22.039720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ad9dd-aa95-42e9-b1cd-9bcd3c73118e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:22.040448 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ad9dd-aa95-42e9-b1cd-9bcd3c73118e service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:22.040799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ad9dd-aa95-42e9-b1cd-9bcd3c73118e service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:22.041140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ad9dd-aa95-42e9-b1cd-9bcd3c73118e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:22.050758 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ad9dd-aa95-42e9-b1cd-9bcd3c73118e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:22.052259 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ad9dd-aa95-42e9-b1cd-9bcd3c73118e service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:22.062084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 893/1792] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:22.064431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ad9dd-aa95-42e9-b1cd-9bcd3c73118e service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:22.064431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e95ad9dd-aa95-42e9-b1cd-9bcd3c73118e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:22.159751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ad9dd-aa95-42e9-b1cd-9bcd3c73118e service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:22.163821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29857c94-3d83-44bb-a089-d5d44d52ae32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:22.164683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857c94-3d83-44bb-a089-d5d44d52ae32 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:22.165432 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857c94-3d83-44bb-a089-d5d44d52ae32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:22.165778 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857c94-3d83-44bb-a089-d5d44d52ae32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:22.176915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29857c94-3d83-44bb-a089-d5d44d52ae32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:22.178399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29857c94-3d83-44bb-a089-d5d44d52ae32 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:22.193434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 894/1793] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:22.199334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba9a6eb1ff9d4af0970503fe85982043'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:22.265911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba9a6eb1ff9d4af0970503fe85982043'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:22.296746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba9a6eb1ff9d4af0970503fe85982043'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:22.395702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba9a6eb1ff9d4af0970503fe85982043'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:22.397509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29857c94-3d83-44bb-a089-d5d44d52ae32 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:22.399059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba9a6eb1ff9d4af0970503fe85982043'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:22.405494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-29857c94-3d83-44bb-a089-d5d44d52ae32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:22.417678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 900/1794] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:21 2022] DELETE /identity/v3/users/ba9a6eb1ff9d4af0970503fe85982043 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:22.434384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29857c94-3d83-44bb-a089-d5d44d52ae32 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:22.644667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a43dc5-6275-4c50-9075-34645ac6ec84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:22.649891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a43dc5-6275-4c50-9075-34645ac6ec84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:22.650283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a43dc5-6275-4c50-9075-34645ac6ec84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:22.650583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a43dc5-6275-4c50-9075-34645ac6ec84 admin admin] PATH_INFO: `/v3/users/58bfff564cd04ccdbac2f3c9d9d294af` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:22.657905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a43dc5-6275-4c50-9075-34645ac6ec84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58bfff564cd04ccdbac2f3c9d9d294af)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:22.664642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a43dc5-6275-4c50-9075-34645ac6ec84 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:23.025103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-20a43dc5-6275-4c50-9075-34645ac6ec84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58bfff564cd04ccdbac2f3c9d9d294af'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:23.038529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-20a43dc5-6275-4c50-9075-34645ac6ec84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58bfff564cd04ccdbac2f3c9d9d294af'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:23.076948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-20a43dc5-6275-4c50-9075-34645ac6ec84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58bfff564cd04ccdbac2f3c9d9d294af'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:23.174125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-20a43dc5-6275-4c50-9075-34645ac6ec84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58bfff564cd04ccdbac2f3c9d9d294af'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:23.176349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:23.178692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-20a43dc5-6275-4c50-9075-34645ac6ec84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58bfff564cd04ccdbac2f3c9d9d294af'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:23.189736 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 895/1795] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:22 2022] DELETE /identity/v3/users/58bfff564cd04ccdbac2f3c9d9d294af => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:23.193160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:23.196086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:23.198469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bbae0-bb70-4258-ab3f-e2b706b9b0f0 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:23.198754 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bbae0-bb70-4258-ab3f-e2b706b9b0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:23.200389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bbae0-bb70-4258-ab3f-e2b706b9b0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:23.216623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252f12b-295b-4176-9dea-0465afcf9981 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:23.239389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a75bbae0-bb70-4258-ab3f-e2b706b9b0f0 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:23.240852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a75bbae0-bb70-4258-ab3f-e2b706b9b0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:23.309105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:23.318873 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:23.320430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:23.321083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:23.321189 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 896/1796] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:23.321700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] PATH_INFO: `/v3/projects/9d04e2c9f3ca43e98eff6a0c4703810e` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:23.326057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d04e2c9f3ca43e98eff6a0c4703810e)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:23.326057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:23.334261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bbae0-bb70-4258-ab3f-e2b706b9b0f0 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:23.335638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a75bbae0-bb70-4258-ab3f-e2b706b9b0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:23.416840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bbae0-bb70-4258-ab3f-e2b706b9b0f0 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:23.421130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f34ea8-158f-4404-9f43-f7ee2bee80c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:23.422142 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f34ea8-158f-4404-9f43-f7ee2bee80c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:23.422469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f34ea8-158f-4404-9f43-f7ee2bee80c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:23.422757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f34ea8-158f-4404-9f43-f7ee2bee80c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:23.435681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f34ea8-158f-4404-9f43-f7ee2bee80c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:23.436916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f34ea8-158f-4404-9f43-f7ee2bee80c2 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:23.447812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 897/1797] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:23.512063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d04e2c9f3ca43e98eff6a0c4703810e', 'request_id': 'req-6eb9917b-3264-4f94-b101-428451711b99'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:23.651763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d04e2c9f3ca43e98eff6a0c4703810e', 'request_id': 'req-6eb9917b-3264-4f94-b101-428451711b99'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:23.663909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d04e2c9f3ca43e98eff6a0c4703810e', 'request_id': 'req-6eb9917b-3264-4f94-b101-428451711b99'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:23.688576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] The token cache is being invalidate because project 9d04e2c9f3ca43e98eff6a0c4703810e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:23.689729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:23.691145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-6eb9917b-3264-4f94-b101-428451711b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:23.695508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 901/1798] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:23 2022] DELETE /identity/v3/projects/9d04e2c9f3ca43e98eff6a0c4703810e => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:23.701901 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284a14c-825f-4fa6-bbe8-98c4a3ea2ee7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:23.702386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284a14c-825f-4fa6-bbe8-98c4a3ea2ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:23.708551 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284a14c-825f-4fa6-bbe8-98c4a3ea2ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:23.737943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6284a14c-825f-4fa6-bbe8-98c4a3ea2ee7 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:23.739445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6284a14c-825f-4fa6-bbe8-98c4a3ea2ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:23.784276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:23.828672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 898/1799] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:23.837936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172d5e-f9d0-46c5-b57e-928db11ce492 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:23.838614 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172d5e-f9d0-46c5-b57e-928db11ce492 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:23.839027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172d5e-f9d0-46c5-b57e-928db11ce492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:23.861917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1e172d5e-f9d0-46c5-b57e-928db11ce492 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:23.862411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1e172d5e-f9d0-46c5-b57e-928db11ce492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:23.878211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 902/1800] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:23.881321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284a14c-825f-4fa6-bbe8-98c4a3ea2ee7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:23.897653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284a14c-825f-4fa6-bbe8-98c4a3ea2ee7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:23.903338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208494d-5a14-4ad7-b766-d0b9d30484f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:23.904005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208494d-5a14-4ad7-b766-d0b9d30484f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:23.904288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208494d-5a14-4ad7-b766-d0b9d30484f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:23.904564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208494d-5a14-4ad7-b766-d0b9d30484f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:23.905400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208494d-5a14-4ad7-b766-d0b9d30484f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:23.905712 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208494d-5a14-4ad7-b766-d0b9d30484f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:23.911062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208494d-5a14-4ad7-b766-d0b9d30484f8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:23.926122 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 899/1801] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:23.931119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e172d5e-f9d0-46c5-b57e-928db11ce492 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:23.942674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e172d5e-f9d0-46c5-b57e-928db11ce492 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:23.948829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73608b9-3337-4135-982b-c8f9ac0e9e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:23.949536 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73608b9-3337-4135-982b-c8f9ac0e9e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:23.949890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73608b9-3337-4135-982b-c8f9ac0e9e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:23.950077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73608b9-3337-4135-982b-c8f9ac0e9e1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:23.950782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73608b9-3337-4135-982b-c8f9ac0e9e1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:23.952742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73608b9-3337-4135-982b-c8f9ac0e9e1d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:24.000794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208494d-5a14-4ad7-b766-d0b9d30484f8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:24.010295 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0208494d-5a14-4ad7-b766-d0b9d30484f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:24.040031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208494d-5a14-4ad7-b766-d0b9d30484f8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:24.050461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:24.051696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.057366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.057366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] PATH_INFO: `/v3/projects/974112776ff744c98676805c8616cc48` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.057366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] RBAC: Authorizing `identity:delete_project(project_id=974112776ff744c98676805c8616cc48)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:24.057366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:24.101894 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 903/1802] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:23 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:24.106792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73608b9-3337-4135-982b-c8f9ac0e9e1d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:24.221634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '974112776ff744c98676805c8616cc48', 'request_id': 'req-01aab4ff-259e-49f5-a2a6-943fb833a40a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:24.223134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73608b9-3337-4135-982b-c8f9ac0e9e1d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:24.230281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '974112776ff744c98676805c8616cc48', 'request_id': 'req-01aab4ff-259e-49f5-a2a6-943fb833a40a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:24.245742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '974112776ff744c98676805c8616cc48', 'request_id': 'req-01aab4ff-259e-49f5-a2a6-943fb833a40a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:24.295757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c8d1a-1ce7-4892-8149-b5ce2db76381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:24.296454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c8d1a-1ce7-4892-8149-b5ce2db76381 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.296792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c8d1a-1ce7-4892-8149-b5ce2db76381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.297133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c8d1a-1ce7-4892-8149-b5ce2db76381 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.297845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c8d1a-1ce7-4892-8149-b5ce2db76381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:24.302215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c8d1a-1ce7-4892-8149-b5ce2db76381 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:24.326199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] The token cache is being invalidate because project 974112776ff744c98676805c8616cc48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:24.326631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:24.327418 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-01aab4ff-259e-49f5-a2a6-943fb833a40a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:24.329486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 900/1803] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:23 2022] DELETE /identity/v3/projects/974112776ff744c98676805c8616cc48 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:24.332724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70ea97-59c9-44cd-813b-d7761ca3e0d5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.333053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70ea97-59c9-44cd-813b-d7761ca3e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.333338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70ea97-59c9-44cd-813b-d7761ca3e0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.362826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-dc70ea97-59c9-44cd-813b-d7761ca3e0d5 None None] MFA Rules not processed for user `f13537780e3a490db7f19451e7be9e43`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:24.364828 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dc70ea97-59c9-44cd-813b-d7761ca3e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:24.421828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 904/1804] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:24 2022] POST /identity/v3/users => generated 285 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:50:24.424218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c8d1a-1ce7-4892-8149-b5ce2db76381 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:24.425617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a65c8d1a-1ce7-4892-8149-b5ce2db76381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:24.438585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c8d1a-1ce7-4892-8149-b5ce2db76381 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:24.439567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 901/1805] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:24.440485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70ea97-59c9-44cd-813b-d7761ca3e0d5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:24.442006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dc70ea97-59c9-44cd-813b-d7761ca3e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:24.445881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:24.446769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.447096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.447333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] PATH_INFO: `/v3/users/4f6975ed169f491a9f9c2add4d414b62` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.448566 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f6975ed169f491a9f9c2add4d414b62)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:24.450744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:24.456914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70ea97-59c9-44cd-813b-d7761ca3e0d5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:24.461809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b195cbb-c3d5-4771-85c2-6b458f693784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:24.462478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b195cbb-c3d5-4771-85c2-6b458f693784 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.462814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b195cbb-c3d5-4771-85c2-6b458f693784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.463082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b195cbb-c3d5-4771-85c2-6b458f693784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.464113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b195cbb-c3d5-4771-85c2-6b458f693784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:24.467088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b195cbb-c3d5-4771-85c2-6b458f693784 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:24.477739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 902/1806] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:24.478891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b195cbb-c3d5-4771-85c2-6b458f693784 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:24.480913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1b195cbb-c3d5-4771-85c2-6b458f693784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:24.497843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b195cbb-c3d5-4771-85c2-6b458f693784 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:24.502854 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a74533-25b7-4bb6-8679-5f5565d700ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:24.503539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a74533-25b7-4bb6-8679-5f5565d700ff admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.503821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a74533-25b7-4bb6-8679-5f5565d700ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.504127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a74533-25b7-4bb6-8679-5f5565d700ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.504840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a74533-25b7-4bb6-8679-5f5565d700ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:24.506857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a74533-25b7-4bb6-8679-5f5565d700ff admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:24.623954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 903/1807] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:24 2022] POST /identity/v3/projects => generated 403 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:24.624820 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a74533-25b7-4bb6-8679-5f5565d700ff admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:24.645893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f6975ed169f491a9f9c2add4d414b62'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:24.662410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f6975ed169f491a9f9c2add4d414b62'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:24.670376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f6975ed169f491a9f9c2add4d414b62'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:24.697561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a74533-25b7-4bb6-8679-5f5565d700ff admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:24.702993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebb24f-5cbb-42e9-b018-d307161949df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:24.703703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebb24f-5cbb-42e9-b018-d307161949df admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.703993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebb24f-5cbb-42e9-b018-d307161949df admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.704273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebb24f-5cbb-42e9-b018-d307161949df admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.705079 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebb24f-5cbb-42e9-b018-d307161949df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:24.706708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebb24f-5cbb-42e9-b018-d307161949df admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:24.719142 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 904/1808] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:24.720093 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebb24f-5cbb-42e9-b018-d307161949df admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:24.730951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f6975ed169f491a9f9c2add4d414b62'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:24.731192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebb24f-5cbb-42e9-b018-d307161949df admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:24.731758 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f6975ed169f491a9f9c2add4d414b62'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:24.736033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ece820-0755-4aa5-b5df-8eb496dfd30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:24.746223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ece820-0755-4aa5-b5df-8eb496dfd30c admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.746223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ece820-0755-4aa5-b5df-8eb496dfd30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.746223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ece820-0755-4aa5-b5df-8eb496dfd30c admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.746223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ece820-0755-4aa5-b5df-8eb496dfd30c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:24.746223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ece820-0755-4aa5-b5df-8eb496dfd30c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:24.746600 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 905/1809] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:24 2022] DELETE /identity/v3/users/4f6975ed169f491a9f9c2add4d414b62 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:24.746600 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:24.746600 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:24.818831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2756a-13c4-4faa-bb59-895d9a68a991 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:24.823947 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0f7ab-411d-4d79-91ea-7deb5e4a7764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:24.825088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0f7ab-411d-4d79-91ea-7deb5e4a7764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.825525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0f7ab-411d-4d79-91ea-7deb5e4a7764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.825923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0f7ab-411d-4d79-91ea-7deb5e4a7764 admin admin] PATH_INFO: `/v3/projects/db7a9603d2d74feeaa4305bac1c682d4/users/c5caf6c8b24746d1bd3c19f17881f325/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.835375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 905/1810] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:24 2022] POST /identity/v3/users => generated 308 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:24.839749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ece820-0755-4aa5-b5df-8eb496dfd30c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:24.841566 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-84ece820-0755-4aa5-b5df-8eb496dfd30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:24.845524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0f7ab-411d-4d79-91ea-7deb5e4a7764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7a9603d2d74feeaa4305bac1c682d4, user_id=c5caf6c8b24746d1bd3c19f17881f325, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:24.848144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0f7ab-411d-4d79-91ea-7deb5e4a7764 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:24.859122 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ece820-0755-4aa5-b5df-8eb496dfd30c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:24.866586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787f5d6-5cd6-4a29-87c3-cb8f37b5d159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:24.867281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787f5d6-5cd6-4a29-87c3-cb8f37b5d159 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.867568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787f5d6-5cd6-4a29-87c3-cb8f37b5d159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.867852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787f5d6-5cd6-4a29-87c3-cb8f37b5d159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.868604 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787f5d6-5cd6-4a29-87c3-cb8f37b5d159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:24.870295 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787f5d6-5cd6-4a29-87c3-cb8f37b5d159 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:24.884750 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 906/1811] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:50:24.889802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787f5d6-5cd6-4a29-87c3-cb8f37b5d159 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:24.901397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787f5d6-5cd6-4a29-87c3-cb8f37b5d159 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:24.906952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370f6f4-e9c5-4a82-861d-b8c154bf4915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:24.907850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370f6f4-e9c5-4a82-861d-b8c154bf4915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.908056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370f6f4-e9c5-4a82-861d-b8c154bf4915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.908404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370f6f4-e9c5-4a82-861d-b8c154bf4915 admin admin] PATH_INFO: `/v3/projects/fe54c0a1dfda4e3fa8d7563cf54fc198/users/f45355714bc44562a5188d31cc9c7514/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.929702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370f6f4-e9c5-4a82-861d-b8c154bf4915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe54c0a1dfda4e3fa8d7563cf54fc198, user_id=f45355714bc44562a5188d31cc9c7514, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:24.933005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370f6f4-e9c5-4a82-861d-b8c154bf4915 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:24.950588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 906/1812] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:24 2022] PUT /identity/v3/projects/db7a9603d2d74feeaa4305bac1c682d4/users/c5caf6c8b24746d1bd3c19f17881f325/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:24.963982 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579d707-27a2-4de6-811b-66c5af48167f None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:24.964756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579d707-27a2-4de6-811b-66c5af48167f None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:24.965555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579d707-27a2-4de6-811b-66c5af48167f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:24.968347 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:25.005438 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f579d707-27a2-4de6-811b-66c5af48167f None None] MFA Rules not processed for user `c5caf6c8b24746d1bd3c19f17881f325`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:25.007293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f579d707-27a2-4de6-811b-66c5af48167f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:25.023388 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 907/1813] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:24 2022] PUT /identity/v3/projects/fe54c0a1dfda4e3fa8d7563cf54fc198/users/f45355714bc44562a5188d31cc9c7514/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:25.028201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370f6f4-e9c5-4a82-861d-b8c154bf4915 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:25.103930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 907/1814] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:25.107242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579d707-27a2-4de6-811b-66c5af48167f None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:25.127282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370f6f4-e9c5-4a82-861d-b8c154bf4915 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:25.135033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b969b-4692-4744-8bea-c385fa69b9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:25.135730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b969b-4692-4744-8bea-c385fa69b9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:25.136014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b969b-4692-4744-8bea-c385fa69b9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:25.136541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b969b-4692-4744-8bea-c385fa69b9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:25.137258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b969b-4692-4744-8bea-c385fa69b9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:25.139599 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b969b-4692-4744-8bea-c385fa69b9d7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:25.153704 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 908/1815] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:25.159385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b969b-4692-4744-8bea-c385fa69b9d7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:25.177369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b969b-4692-4744-8bea-c385fa69b9d7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:25.185660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2f65a-c152-4de2-b4e5-cd34143b2cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:25.186570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2f65a-c152-4de2-b4e5-cd34143b2cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:25.187079 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2f65a-c152-4de2-b4e5-cd34143b2cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:25.187645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2f65a-c152-4de2-b4e5-cd34143b2cd6 admin admin] PATH_INFO: `/v3/projects/fe54c0a1dfda4e3fa8d7563cf54fc198/users/f45355714bc44562a5188d31cc9c7514/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:25.190784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2f65a-c152-4de2-b4e5-cd34143b2cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe54c0a1dfda4e3fa8d7563cf54fc198, user_id=f45355714bc44562a5188d31cc9c7514, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:25.194622 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2f65a-c152-4de2-b4e5-cd34143b2cd6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:25.210917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579d707-27a2-4de6-811b-66c5af48167f None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:25.218733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 909/1816] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:25 2022] PUT /identity/v3/projects/fe54c0a1dfda4e3fa8d7563cf54fc198/users/f45355714bc44562a5188d31cc9c7514/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:25.301653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d20bf-70fc-489b-b9b5-281d7972c538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:25.303010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d20bf-70fc-489b-b9b5-281d7972c538 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:25.303346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d20bf-70fc-489b-b9b5-281d7972c538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:25.303635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d20bf-70fc-489b-b9b5-281d7972c538 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:25.304548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d20bf-70fc-489b-b9b5-281d7972c538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:25.306349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d20bf-70fc-489b-b9b5-281d7972c538 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:25.344052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2f65a-c152-4de2-b4e5-cd34143b2cd6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:25.346029 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0da2f65a-c152-4de2-b4e5-cd34143b2cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:25.381694 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2f65a-c152-4de2-b4e5-cd34143b2cd6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:25.386739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:25.389848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:25.390163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:25.390450 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] PATH_INFO: `/v3/projects/4a8072ad73f54e0292c916118ac4c78d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:25.393173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8072ad73f54e0292c916118ac4c78d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:25.395367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:25.424377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:25.512346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 908/1817] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:25 2022] POST /identity/v3/projects => generated 367 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:25.512346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d20bf-70fc-489b-b9b5-281d7972c538 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:25.566032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d20bf-70fc-489b-b9b5-281d7972c538 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:25.571105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5742c52-b935-408a-9904-ee02b72b05bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:25.571867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5742c52-b935-408a-9904-ee02b72b05bc admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:25.571980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5742c52-b935-408a-9904-ee02b72b05bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:25.572176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5742c52-b935-408a-9904-ee02b72b05bc admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:25.572892 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5742c52-b935-408a-9904-ee02b72b05bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:25.575369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5742c52-b935-408a-9904-ee02b72b05bc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:25.595291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8072ad73f54e0292c916118ac4c78d', 'request_id': 'req-92295836-00da-43b6-b408-5e2b47553b4d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:25.602881 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8072ad73f54e0292c916118ac4c78d', 'request_id': 'req-92295836-00da-43b6-b408-5e2b47553b4d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:25.609602 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8072ad73f54e0292c916118ac4c78d', 'request_id': 'req-92295836-00da-43b6-b408-5e2b47553b4d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:25.713464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 909/1818] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:25 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:50:25.717343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5742c52-b935-408a-9904-ee02b72b05bc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:25.727826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5742c52-b935-408a-9904-ee02b72b05bc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:25.757552 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] The token cache is being invalidate because project 4a8072ad73f54e0292c916118ac4c78d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:25.757552 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:25.759924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-92295836-00da-43b6-b408-5e2b47553b4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:25.762516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 910/1819] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:25 2022] DELETE /identity/v3/projects/4a8072ad73f54e0292c916118ac4c78d => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:25.769466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fdac1-b91a-42ec-ba34-7fabf99e5a8d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:25.770387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fdac1-b91a-42ec-ba34-7fabf99e5a8d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:25.770674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fdac1-b91a-42ec-ba34-7fabf99e5a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:25.793132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd6ee1-41ae-485f-a911-4606a98276cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:25.793385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-010fdac1-b91a-42ec-ba34-7fabf99e5a8d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:25.793488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd6ee1-41ae-485f-a911-4606a98276cc admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:25.793488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd6ee1-41ae-485f-a911-4606a98276cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:25.800555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd6ee1-41ae-485f-a911-4606a98276cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:25.800555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd6ee1-41ae-485f-a911-4606a98276cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:25.800740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-010fdac1-b91a-42ec-ba34-7fabf99e5a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:25.800819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd6ee1-41ae-485f-a911-4606a98276cc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:25.803525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 911/1820] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:25.811057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 910/1821] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:50:25.811571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8f757-bbe8-42d0-9e38-7eabae70c93f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:25.811864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8f757-bbe8-42d0-9e38-7eabae70c93f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:25.812158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8f757-bbe8-42d0-9e38-7eabae70c93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:25.815787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd6ee1-41ae-485f-a911-4606a98276cc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:25.817815 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ebbd6ee1-41ae-485f-a911-4606a98276cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:25.833798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd6ee1-41ae-485f-a911-4606a98276cc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:25.834053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-dee8f757-bbe8-42d0-9e38-7eabae70c93f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:25.835383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dee8f757-bbe8-42d0-9e38-7eabae70c93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:25.840298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265afaf-4854-4b48-a4f7-0806b70a6f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:25.841330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265afaf-4854-4b48-a4f7-0806b70a6f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:25.841673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265afaf-4854-4b48-a4f7-0806b70a6f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:25.842045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265afaf-4854-4b48-a4f7-0806b70a6f4f admin admin] PATH_INFO: `/v3/projects/dbd35d396f1147b396ac0f8e6b493714/users/7474fdb420124b3a9651d16c681138b3/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:25.846160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 912/1822] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:25.851169 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8f757-bbe8-42d0-9e38-7eabae70c93f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:25.861269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8f757-bbe8-42d0-9e38-7eabae70c93f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:25.863470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265afaf-4854-4b48-a4f7-0806b70a6f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd35d396f1147b396ac0f8e6b493714, user_id=7474fdb420124b3a9651d16c681138b3, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:25.866356 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265afaf-4854-4b48-a4f7-0806b70a6f4f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:25.866466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420c679-0c9d-42ed-9671-649a6a1b0379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:25.867047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c679-0c9d-42ed-9671-649a6a1b0379 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:25.867360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c679-0c9d-42ed-9671-649a6a1b0379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:25.867720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420c679-0c9d-42ed-9671-649a6a1b0379 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:25.868411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420c679-0c9d-42ed-9671-649a6a1b0379 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:25.868721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420c679-0c9d-42ed-9671-649a6a1b0379 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:25.870364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420c679-0c9d-42ed-9671-649a6a1b0379 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:25.884115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 913/1823] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:25.888320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420c679-0c9d-42ed-9671-649a6a1b0379 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:25.900686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420c679-0c9d-42ed-9671-649a6a1b0379 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:25.905703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaed2c2-3135-486d-a7f3-75a14d835505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:25.906467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaed2c2-3135-486d-a7f3-75a14d835505 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:25.906761 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaed2c2-3135-486d-a7f3-75a14d835505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:25.907036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaed2c2-3135-486d-a7f3-75a14d835505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:25.907735 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaed2c2-3135-486d-a7f3-75a14d835505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:25.909734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaed2c2-3135-486d-a7f3-75a14d835505 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:25.954916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 911/1824] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:25 2022] PUT /identity/v3/projects/dbd35d396f1147b396ac0f8e6b493714/users/7474fdb420124b3a9651d16c681138b3/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:25.958990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265afaf-4854-4b48-a4f7-0806b70a6f4f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:26.023701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265afaf-4854-4b48-a4f7-0806b70a6f4f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:26.025937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 914/1825] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:25 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:26.030300 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaed2c2-3135-486d-a7f3-75a14d835505 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:26.102593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27a174-7299-49e1-aef8-f761a6280699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:26.103284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27a174-7299-49e1-aef8-f761a6280699 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:26.103570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27a174-7299-49e1-aef8-f761a6280699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:26.104099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27a174-7299-49e1-aef8-f761a6280699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:26.104869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27a174-7299-49e1-aef8-f761a6280699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:26.106519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27a174-7299-49e1-aef8-f761a6280699 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:26.120763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 912/1826] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:50:26.120985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaed2c2-3135-486d-a7f3-75a14d835505 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:26.122004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27a174-7299-49e1-aef8-f761a6280699 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:26.124813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d62e8-32c6-49ed-93fd-52c27e8f1229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:26.128543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d62e8-32c6-49ed-93fd-52c27e8f1229 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:26.128543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d62e8-32c6-49ed-93fd-52c27e8f1229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:26.128543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d62e8-32c6-49ed-93fd-52c27e8f1229 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:26.128543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d62e8-32c6-49ed-93fd-52c27e8f1229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:26.130210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d62e8-32c6-49ed-93fd-52c27e8f1229 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:26.133199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27a174-7299-49e1-aef8-f761a6280699 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:26.138927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619d247-0fe7-4820-9076-354850550d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:26.139863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619d247-0fe7-4820-9076-354850550d09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:26.140224 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619d247-0fe7-4820-9076-354850550d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:26.140505 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619d247-0fe7-4820-9076-354850550d09 admin admin] PATH_INFO: `/v3/projects/dbd35d396f1147b396ac0f8e6b493714/users/7474fdb420124b3a9651d16c681138b3/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:26.142590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619d247-0fe7-4820-9076-354850550d09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd35d396f1147b396ac0f8e6b493714, user_id=7474fdb420124b3a9651d16c681138b3, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:26.145869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619d247-0fe7-4820-9076-354850550d09 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:26.156775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 913/1827] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:26 2022] PUT /identity/v3/projects/dbd35d396f1147b396ac0f8e6b493714/users/7474fdb420124b3a9651d16c681138b3/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:26.167148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d6cf4-cb9a-4bd1-8938-a718aac762a1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:26.167582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d6cf4-cb9a-4bd1-8938-a718aac762a1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:26.167877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d6cf4-cb9a-4bd1-8938-a718aac762a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:26.196689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-105d6cf4-cb9a-4bd1-8938-a718aac762a1 None None] MFA Rules not processed for user `7474fdb420124b3a9651d16c681138b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:26.198028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-105d6cf4-cb9a-4bd1-8938-a718aac762a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:26.240337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 915/1828] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:26 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:50:26.244621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d62e8-32c6-49ed-93fd-52c27e8f1229 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:26.259674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d62e8-32c6-49ed-93fd-52c27e8f1229 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:26.281656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 914/1829] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:26.293628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d6cf4-cb9a-4bd1-8938-a718aac762a1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:26.295158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-105d6cf4-cb9a-4bd1-8938-a718aac762a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:26.314273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:26.355497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f340e7-ce69-4c8c-ac58-2bceae1e95f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:26.356178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f340e7-ce69-4c8c-ac58-2bceae1e95f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:26.356462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f340e7-ce69-4c8c-ac58-2bceae1e95f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:26.356734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f340e7-ce69-4c8c-ac58-2bceae1e95f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:26.357754 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f340e7-ce69-4c8c-ac58-2bceae1e95f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:26.359582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f340e7-ce69-4c8c-ac58-2bceae1e95f7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:26.374260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 916/1830] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:50:26.375372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f340e7-ce69-4c8c-ac58-2bceae1e95f7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:26.383967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d6cf4-cb9a-4bd1-8938-a718aac762a1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:26.387464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f340e7-ce69-4c8c-ac58-2bceae1e95f7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:26.387663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a97f6-7764-4402-b5bd-e7dfdba6faa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:26.388287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a97f6-7764-4402-b5bd-e7dfdba6faa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:26.388658 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a97f6-7764-4402-b5bd-e7dfdba6faa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:26.389161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a97f6-7764-4402-b5bd-e7dfdba6faa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:26.392991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58e232-58be-4b84-9ea8-bf8bb9656f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:26.393727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58e232-58be-4b84-9ea8-bf8bb9656f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:26.394072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58e232-58be-4b84-9ea8-bf8bb9656f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:26.394474 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58e232-58be-4b84-9ea8-bf8bb9656f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:26.395324 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58e232-58be-4b84-9ea8-bf8bb9656f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:26.397034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58e232-58be-4b84-9ea8-bf8bb9656f1d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:26.400800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a97f6-7764-4402-b5bd-e7dfdba6faa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:26.402690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a97f6-7764-4402-b5bd-e7dfdba6faa0 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:26.412549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 917/1831] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:26.415198 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58e232-58be-4b84-9ea8-bf8bb9656f1d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:26.421085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 915/1832] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:26.441458 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58e232-58be-4b84-9ea8-bf8bb9656f1d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:26.456792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b385f-1e1c-471d-b10e-ad22f4dc8840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:26.458889 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b385f-1e1c-471d-b10e-ad22f4dc8840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:26.459172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b385f-1e1c-471d-b10e-ad22f4dc8840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:26.460610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b385f-1e1c-471d-b10e-ad22f4dc8840 admin admin] PATH_INFO: `/v3/projects/234f4c6ce6a84a37b1902871409807f3/users/67abe67c7af84c38998f10036c1bada4/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:26.494724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b385f-1e1c-471d-b10e-ad22f4dc8840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=234f4c6ce6a84a37b1902871409807f3, user_id=67abe67c7af84c38998f10036c1bada4, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:26.497818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b385f-1e1c-471d-b10e-ad22f4dc8840 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:26.636650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 918/1833] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:26 2022] PUT /identity/v3/projects/234f4c6ce6a84a37b1902871409807f3/users/67abe67c7af84c38998f10036c1bada4/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:26.648034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2217a-de8b-4e64-92e2-5fc560afd5fa None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:26.648540 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2217a-de8b-4e64-92e2-5fc560afd5fa None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:26.648641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2217a-de8b-4e64-92e2-5fc560afd5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:26.682449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3bf2217a-de8b-4e64-92e2-5fc560afd5fa None None] MFA Rules not processed for user `67abe67c7af84c38998f10036c1bada4`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:26.683666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3bf2217a-de8b-4e64-92e2-5fc560afd5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:26.752209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 916/1834] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:26.769366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b385f-1e1c-471d-b10e-ad22f4dc8840 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:26.835922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b385f-1e1c-471d-b10e-ad22f4dc8840 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:26.839940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a181d0a3-41b1-4aa9-8fe1-8f4f04a35cbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:26.840590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181d0a3-41b1-4aa9-8fe1-8f4f04a35cbe service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:26.840902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181d0a3-41b1-4aa9-8fe1-8f4f04a35cbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:26.841191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a181d0a3-41b1-4aa9-8fe1-8f4f04a35cbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:26.850846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181d0a3-41b1-4aa9-8fe1-8f4f04a35cbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:26.852313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a181d0a3-41b1-4aa9-8fe1-8f4f04a35cbe service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:26.862210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 919/1835] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:26.871779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30192d2a-5b0f-4bd3-84a2-b04b165c9132 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:26.872124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30192d2a-5b0f-4bd3-84a2-b04b165c9132 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:26.872539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30192d2a-5b0f-4bd3-84a2-b04b165c9132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:26.946016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-30192d2a-5b0f-4bd3-84a2-b04b165c9132 None None] MFA Rules not processed for user `f45355714bc44562a5188d31cc9c7514`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:26.946016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-30192d2a-5b0f-4bd3-84a2-b04b165c9132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:27.023423 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:27.033640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 917/1836] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:26 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:27.886978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5879920-988a-4d87-87b7-5a261fab8bf1 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:27.887969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5879920-988a-4d87-87b7-5a261fab8bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:27.888463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5879920-988a-4d87-87b7-5a261fab8bf1 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:27.890100 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 920/1837] 10.222.0.88 () {54 vars in 921 bytes} [Mon Jul 18 16:50:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:50:27.917397 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2e416-afa7-429b-8c15-5c047a42b28a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:27.917890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2e416-afa7-429b-8c15-5c047a42b28a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:27.918006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2e416-afa7-429b-8c15-5c047a42b28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:27.942986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e5e2e416-afa7-429b-8c15-5c047a42b28a None None] MFA Rules not processed for user `22bd4dab1fcd4ad4a8be886646d89242`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:27.944578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e5e2e416-afa7-429b-8c15-5c047a42b28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:28.038884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 918/1838] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Jul 18 16:50:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:28.046381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:28.135044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5879920-988a-4d87-87b7-5a261fab8bf1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:28.138085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f5879920-988a-4d87-87b7-5a261fab8bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:28.162257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:28.166469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5879920-988a-4d87-87b7-5a261fab8bf1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:28.174651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6a141-17e5-4f33-9316-54a918b939f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:28.175460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6a141-17e5-4f33-9316-54a918b939f5 service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:28.175808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6a141-17e5-4f33-9316-54a918b939f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:28.176185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6a141-17e5-4f33-9316-54a918b939f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:28.189123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6a141-17e5-4f33-9316-54a918b939f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:28.193195 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e6a141-17e5-4f33-9316-54a918b939f5 service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:28.205241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 921/1839] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:29.025206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:29.117819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2e416-afa7-429b-8c15-5c047a42b28a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:29.119288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e5e2e416-afa7-429b-8c15-5c047a42b28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:29.199045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2e416-afa7-429b-8c15-5c047a42b28a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:29.204399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b3fdb-ae12-415f-a340-44f2b578e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:29.205445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b3fdb-ae12-415f-a340-44f2b578e471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:29.205766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b3fdb-ae12-415f-a340-44f2b578e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:29.206071 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b3fdb-ae12-415f-a340-44f2b578e471 admin admin] PATH_INFO: `/v3/users/f13537780e3a490db7f19451e7be9e43` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:29.207372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b3fdb-ae12-415f-a340-44f2b578e471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f13537780e3a490db7f19451e7be9e43)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:29.209571 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b3fdb-ae12-415f-a340-44f2b578e471 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:29.429702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-704b3fdb-ae12-415f-a340-44f2b578e471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f13537780e3a490db7f19451e7be9e43'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:29.451378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-704b3fdb-ae12-415f-a340-44f2b578e471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f13537780e3a490db7f19451e7be9e43'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:29.471145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-704b3fdb-ae12-415f-a340-44f2b578e471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f13537780e3a490db7f19451e7be9e43'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:29.537258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-704b3fdb-ae12-415f-a340-44f2b578e471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f13537780e3a490db7f19451e7be9e43'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:29.539044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-704b3fdb-ae12-415f-a340-44f2b578e471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f13537780e3a490db7f19451e7be9e43'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:29.548036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 919/1840] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:29 2022] DELETE /identity/v3/users/f13537780e3a490db7f19451e7be9e43 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:29.726157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:30.228723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:30.414076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6a141-17e5-4f33-9316-54a918b939f5 service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:30.427214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6a141-17e5-4f33-9316-54a918b939f5 service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:30.468325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b6d82-eaf3-4787-8422-b26dd9416f60 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:30.468825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b6d82-eaf3-4787-8422-b26dd9416f60 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:30.469149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b6d82-eaf3-4787-8422-b26dd9416f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:30.513311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:30.514374 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:30.522674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:30.522674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] PATH_INFO: `/v3/projects/31f43a812a4548f0b11effb0ef1dec1e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:30.522674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31f43a812a4548f0b11effb0ef1dec1e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:30.522674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:30.579675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6e3b6d82-eaf3-4787-8422-b26dd9416f60 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:30.581121 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6e3b6d82-eaf3-4787-8422-b26dd9416f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:30.648181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:30.656394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31f43a812a4548f0b11effb0ef1dec1e', 'request_id': 'req-fcd3663c-71c6-473b-aa22-70bc94e4cba5'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:30.666126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31f43a812a4548f0b11effb0ef1dec1e', 'request_id': 'req-fcd3663c-71c6-473b-aa22-70bc94e4cba5'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:30.677031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31f43a812a4548f0b11effb0ef1dec1e', 'request_id': 'req-fcd3663c-71c6-473b-aa22-70bc94e4cba5'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:30.716442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 920/1841] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:50:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:30.718140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b6d82-eaf3-4787-8422-b26dd9416f60 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:30.719961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6e3b6d82-eaf3-4787-8422-b26dd9416f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:30.824963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b6d82-eaf3-4787-8422-b26dd9416f60 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:30.831731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] The token cache is being invalidate because project 31f43a812a4548f0b11effb0ef1dec1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:30.832003 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:30.832977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:30.835160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 922/1842] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:30 2022] DELETE /identity/v3/projects/31f43a812a4548f0b11effb0ef1dec1e => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:30.839437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:30.841094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:30.927521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3663c-71c6-473b-aa22-70bc94e4cba5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:30.946481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1baad2b-b089-4db2-86a7-f59907a42a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:30.947982 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1baad2b-b089-4db2-86a7-f59907a42a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:30.949236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1baad2b-b089-4db2-86a7-f59907a42a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:30.950979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a3a33-af4d-496e-884b-1ebd69699dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:30.951384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1baad2b-b089-4db2-86a7-f59907a42a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:30.951976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a3a33-af4d-496e-884b-1ebd69699dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:30.952264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a3a33-af4d-496e-884b-1ebd69699dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:30.952597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a3a33-af4d-496e-884b-1ebd69699dec admin admin] PATH_INFO: `/v3/users/f45355714bc44562a5188d31cc9c7514` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:30.956466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c1baad2b-b089-4db2-86a7-f59907a42a2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:30.958149 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a3a33-af4d-496e-884b-1ebd69699dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=f45355714bc44562a5188d31cc9c7514)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:30.965804 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a3a33-af4d-496e-884b-1ebd69699dec admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:31.230771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b29a3a33-af4d-496e-884b-1ebd69699dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f45355714bc44562a5188d31cc9c7514'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:31.237436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1baad2b-b089-4db2-86a7-f59907a42a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:31.239565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1baad2b-b089-4db2-86a7-f59907a42a2b service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:31.239684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b29a3a33-af4d-496e-884b-1ebd69699dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f45355714bc44562a5188d31cc9c7514'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:31.253482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b29a3a33-af4d-496e-884b-1ebd69699dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f45355714bc44562a5188d31cc9c7514'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:31.277919 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 921/1843] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:31.279110 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1baad2b-b089-4db2-86a7-f59907a42a2b service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:31.281018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c1baad2b-b089-4db2-86a7-f59907a42a2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:31.320836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1baad2b-b089-4db2-86a7-f59907a42a2b service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:31.326500 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:31.333680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:31.334541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:31.334831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:31.335113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] PATH_INFO: `/v3/users/c5caf6c8b24746d1bd3c19f17881f325` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:31.340623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5caf6c8b24746d1bd3c19f17881f325)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:31.342830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:31.364321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b29a3a33-af4d-496e-884b-1ebd69699dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f45355714bc44562a5188d31cc9c7514'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:31.364993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b29a3a33-af4d-496e-884b-1ebd69699dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f45355714bc44562a5188d31cc9c7514'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:31.378546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 923/1844] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:30 2022] DELETE /identity/v3/users/f45355714bc44562a5188d31cc9c7514 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:31.380090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b091ff-5e00-48fd-bdad-15711b38a1b9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:31.380379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b091ff-5e00-48fd-bdad-15711b38a1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:31.380762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b091ff-5e00-48fd-bdad-15711b38a1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:31.427261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-05b091ff-5e00-48fd-bdad-15711b38a1b9 None None] MFA Rules not processed for user `de8484c3b114467ea59bebb2460ed546`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:31.429447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-05b091ff-5e00-48fd-bdad-15711b38a1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:31.538733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 924/1845] 10.222.0.66 () {58 vars in 1015 bytes} [Mon Jul 18 16:50:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:31.540025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b091ff-5e00-48fd-bdad-15711b38a1b9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:31.543009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-05b091ff-5e00-48fd-bdad-15711b38a1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:31.593491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5caf6c8b24746d1bd3c19f17881f325'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:31.633280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5caf6c8b24746d1bd3c19f17881f325'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:31.653993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5caf6c8b24746d1bd3c19f17881f325'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:31.733099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5caf6c8b24746d1bd3c19f17881f325'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:31.734012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5caf6c8b24746d1bd3c19f17881f325'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:31.736724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b091ff-5e00-48fd-bdad-15711b38a1b9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:31.746409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fd8fc-0215-456a-8002-01330ed6bcf1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:31.749999 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fd8fc-0215-456a-8002-01330ed6bcf1 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:31.750857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 922/1846] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:31 2022] DELETE /identity/v3/users/c5caf6c8b24746d1bd3c19f17881f325 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:31.752064 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:31.752186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fd8fc-0215-456a-8002-01330ed6bcf1 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:31.757561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fd8fc-0215-456a-8002-01330ed6bcf1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:31.757561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-313fd8fc-0215-456a-8002-01330ed6bcf1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:31.760696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:31.870216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201589d1-d5bb-4800-a205-6bfce3023633 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:31.877396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed451b43-b0a6-4e93-b8bd-d987a6123166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:31.878365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed451b43-b0a6-4e93-b8bd-d987a6123166 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:31.879768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed451b43-b0a6-4e93-b8bd-d987a6123166 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:31.880904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed451b43-b0a6-4e93-b8bd-d987a6123166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:31.896120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ed451b43-b0a6-4e93-b8bd-d987a6123166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:31.903110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fd8fc-0215-456a-8002-01330ed6bcf1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:31.904741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313fd8fc-0215-456a-8002-01330ed6bcf1 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:31.909695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed451b43-b0a6-4e93-b8bd-d987a6123166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:31.911440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed451b43-b0a6-4e93-b8bd-d987a6123166 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:31.933151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 923/1847] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:31.934401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed451b43-b0a6-4e93-b8bd-d987a6123166 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:31.944401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 925/1848] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:50:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:50:31.944401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199149-fa82-4b01-ae83-f68c3b1928e7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:31.944401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199149-fa82-4b01-ae83-f68c3b1928e7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:31.944401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af199149-fa82-4b01-ae83-f68c3b1928e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:31.945996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ed451b43-b0a6-4e93-b8bd-d987a6123166 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:32.015768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-af199149-fa82-4b01-ae83-f68c3b1928e7 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:32.024945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-af199149-fa82-4b01-ae83-f68c3b1928e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:32.144391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed451b43-b0a6-4e93-b8bd-d987a6123166 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:32.151752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:32.152835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:32.153295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:32.153739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] PATH_INFO: `/v3/projects/fe54c0a1dfda4e3fa8d7563cf54fc198` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:32.155183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe54c0a1dfda4e3fa8d7563cf54fc198)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:32.157802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:32.216152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 926/1849] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:32.250949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af199149-fa82-4b01-ae83-f68c3b1928e7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:32.269065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af199149-fa82-4b01-ae83-f68c3b1928e7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:32.281670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421204b7-fd09-4fab-8825-a0a1c66dc463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:32.282643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421204b7-fd09-4fab-8825-a0a1c66dc463 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:32.283050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421204b7-fd09-4fab-8825-a0a1c66dc463 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:32.283418 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421204b7-fd09-4fab-8825-a0a1c66dc463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:32.312327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421204b7-fd09-4fab-8825-a0a1c66dc463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:32.317984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421204b7-fd09-4fab-8825-a0a1c66dc463 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:32.319945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe54c0a1dfda4e3fa8d7563cf54fc198', 'request_id': 'req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:32.365859 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 927/1850] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:32.442589 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe54c0a1dfda4e3fa8d7563cf54fc198', 'request_id': 'req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:32.458926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe54c0a1dfda4e3fa8d7563cf54fc198', 'request_id': 'req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:32.506886 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] The token cache is being invalidate because project fe54c0a1dfda4e3fa8d7563cf54fc198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:32.507591 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:32.510427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ba6883c-ff0b-4d85-af6b-8ac734d31b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:32.512191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 924/1851] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:31 2022] DELETE /identity/v3/projects/fe54c0a1dfda4e3fa8d7563cf54fc198 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:32.517832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d6071-f540-4c26-92d8-de1d7d73feba None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:32.518415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d6071-f540-4c26-92d8-de1d7d73feba None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:32.518733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d6071-f540-4c26-92d8-de1d7d73feba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:32.576658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-555d6071-f540-4c26-92d8-de1d7d73feba None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:32.578889 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-555d6071-f540-4c26-92d8-de1d7d73feba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:32.642629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:32.684957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 928/1852] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:32.690716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9839d-2622-4155-8efb-08b73e96bcc1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:32.692538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9839d-2622-4155-8efb-08b73e96bcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:32.693857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9839d-2622-4155-8efb-08b73e96bcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:32.724078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-34b9839d-2622-4155-8efb-08b73e96bcc1 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:32.727489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-34b9839d-2622-4155-8efb-08b73e96bcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:32.743721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 925/1853] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:32.746826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d6071-f540-4c26-92d8-de1d7d73feba None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:32.763883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d6071-f540-4c26-92d8-de1d7d73feba None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:32.786654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bf101-6020-474c-bdfa-f841f2340158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:32.787563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bf101-6020-474c-bdfa-f841f2340158 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:32.787863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bf101-6020-474c-bdfa-f841f2340158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:32.788150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bf101-6020-474c-bdfa-f841f2340158 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:32.789091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bf101-6020-474c-bdfa-f841f2340158 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:32.789324 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bf101-6020-474c-bdfa-f841f2340158 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:32.791026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bf101-6020-474c-bdfa-f841f2340158 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:32.821332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 929/1854] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:32.826022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9839d-2622-4155-8efb-08b73e96bcc1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:32.840933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9839d-2622-4155-8efb-08b73e96bcc1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:32.855829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af9d75-a97e-4467-8c5b-4f0acad55d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:32.856513 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af9d75-a97e-4467-8c5b-4f0acad55d8d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:32.856999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af9d75-a97e-4467-8c5b-4f0acad55d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:32.857425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af9d75-a97e-4467-8c5b-4f0acad55d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:32.858289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af9d75-a97e-4467-8c5b-4f0acad55d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:32.860733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af9d75-a97e-4467-8c5b-4f0acad55d8d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:33.011209 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bf101-6020-474c-bdfa-f841f2340158 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:33.018340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 926/1855] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:32 2022] POST /identity/v3/projects => generated 361 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:33.023179 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bf101-6020-474c-bdfa-f841f2340158 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:33.164073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a27b8-3601-4f7c-b214-a873393ea0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:33.168084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a27b8-3601-4f7c-b214-a873393ea0ec admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:33.168084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a27b8-3601-4f7c-b214-a873393ea0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:33.168084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a27b8-3601-4f7c-b214-a873393ea0ec admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:33.168084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a27b8-3601-4f7c-b214-a873393ea0ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:33.170203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a27b8-3601-4f7c-b214-a873393ea0ec admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:33.337312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 930/1856] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:33 2022] POST /identity/v3/users => generated 281 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:33.341920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af9d75-a97e-4467-8c5b-4f0acad55d8d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:33.365135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af9d75-a97e-4467-8c5b-4f0acad55d8d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:33.383800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cba5c1-8832-4e8c-ace2-3460260deb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:33.384734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba5c1-8832-4e8c-ace2-3460260deb3d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:33.389311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba5c1-8832-4e8c-ace2-3460260deb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:33.389645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba5c1-8832-4e8c-ace2-3460260deb3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:33.390538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cba5c1-8832-4e8c-ace2-3460260deb3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:33.393621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cba5c1-8832-4e8c-ace2-3460260deb3d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:33.548321 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 927/1857] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:33.550522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a27b8-3601-4f7c-b214-a873393ea0ec admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:33.577789 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a27b8-3601-4f7c-b214-a873393ea0ec admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:33.593531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e467f-6051-4e62-a097-23977f065c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:33.594429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e467f-6051-4e62-a097-23977f065c74 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:33.594868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e467f-6051-4e62-a097-23977f065c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:33.595284 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e467f-6051-4e62-a097-23977f065c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:33.596202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e467f-6051-4e62-a097-23977f065c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:33.598188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e467f-6051-4e62-a097-23977f065c74 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:33.622441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 931/1858] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:33.626989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cba5c1-8832-4e8c-ace2-3460260deb3d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:33.638761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cba5c1-8832-4e8c-ace2-3460260deb3d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:33.656382 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7be47-44ac-42e4-b95a-98766a995373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:33.663417 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e467f-6051-4e62-a097-23977f065c74 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:33.668820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7be47-44ac-42e4-b95a-98766a995373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:33.669025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-222e467f-6051-4e62-a097-23977f065c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:33.669125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7be47-44ac-42e4-b95a-98766a995373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:33.669125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7be47-44ac-42e4-b95a-98766a995373 admin admin] PATH_INFO: `/v3/projects/81bc6971b06b440182be7c4f31b9d565/users/ed634dbd0a514cb08019d9dc08c79362/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:33.726565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e467f-6051-4e62-a097-23977f065c74 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:33.726565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:33.726565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:33.726565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:33.727926 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] PATH_INFO: `/v3/users/e2ca05736b6e46a0b3d3094981ce22d8` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:33.728023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7be47-44ac-42e4-b95a-98766a995373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bc6971b06b440182be7c4f31b9d565, user_id=ed634dbd0a514cb08019d9dc08c79362, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:33.728121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ca05736b6e46a0b3d3094981ce22d8)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:33.728121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:33.731946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7be47-44ac-42e4-b95a-98766a995373 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:34.175557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 928/1859] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:33 2022] PUT /identity/v3/projects/81bc6971b06b440182be7c4f31b9d565/users/ed634dbd0a514cb08019d9dc08c79362/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:34.186375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f90d6-6b97-432f-81fb-ec78487709c5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:34.193139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f90d6-6b97-432f-81fb-ec78487709c5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:34.193139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f90d6-6b97-432f-81fb-ec78487709c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:34.255679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a83f90d6-6b97-432f-81fb-ec78487709c5 None None] MFA Rules not processed for user `ed634dbd0a514cb08019d9dc08c79362`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:34.255679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a83f90d6-6b97-432f-81fb-ec78487709c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:34.331741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ca05736b6e46a0b3d3094981ce22d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:34.356968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ca05736b6e46a0b3d3094981ce22d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:34.437721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ca05736b6e46a0b3d3094981ce22d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:34.504970 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 929/1860] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:34.522629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f90d6-6b97-432f-81fb-ec78487709c5 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:34.530381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a83f90d6-6b97-432f-81fb-ec78487709c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:34.593909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ca05736b6e46a0b3d3094981ce22d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:34.595543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ca05736b6e46a0b3d3094981ce22d8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:34.610635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 932/1861] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:33 2022] DELETE /identity/v3/users/e2ca05736b6e46a0b3d3094981ce22d8 => generated 0 bytes in 942 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:34.610635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:34.610635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:34.655636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:34.737777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf669e7-4988-4b4b-9416-c115946ed8c3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:34.743884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f90d6-6b97-432f-81fb-ec78487709c5 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:34.748670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5f047-a62b-492c-a05e-ebb888a2a7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:34.749529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f047-a62b-492c-a05e-ebb888a2a7ed admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:34.749914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f047-a62b-492c-a05e-ebb888a2a7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:34.750499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f047-a62b-492c-a05e-ebb888a2a7ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:34.751535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5f047-a62b-492c-a05e-ebb888a2a7ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:34.754387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5f047-a62b-492c-a05e-ebb888a2a7ed admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:34.760260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:34.761184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:34.768472 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:34.768472 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] PATH_INFO: `/v3/projects/db7a9603d2d74feeaa4305bac1c682d4` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:34.768472 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db7a9603d2d74feeaa4305bac1c682d4)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:34.772443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:34.856383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 933/1862] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:34 2022] POST /identity/v3/projects => generated 365 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:34.858365 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170b494-7cb2-418b-a188-bcfd1614d052 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:34.858875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170b494-7cb2-418b-a188-bcfd1614d052 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:34.859333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170b494-7cb2-418b-a188-bcfd1614d052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:34.896230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5170b494-7cb2-418b-a188-bcfd1614d052 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:34.897968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5170b494-7cb2-418b-a188-bcfd1614d052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:34.972176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db7a9603d2d74feeaa4305bac1c682d4', 'request_id': 'req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:35.017642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 934/1863] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:35.018590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170b494-7cb2-418b-a188-bcfd1614d052 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:35.030121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170b494-7cb2-418b-a188-bcfd1614d052 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:35.084209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db7a9603d2d74feeaa4305bac1c682d4', 'request_id': 'req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:35.092695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db7a9603d2d74feeaa4305bac1c682d4', 'request_id': 'req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:35.111514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b11a7-9de9-4f5f-8ee5-3e8c79833406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:35.112442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b11a7-9de9-4f5f-8ee5-3e8c79833406 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.112904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b11a7-9de9-4f5f-8ee5-3e8c79833406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.113379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b11a7-9de9-4f5f-8ee5-3e8c79833406 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.113971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] The token cache is being invalidate because project db7a9603d2d74feeaa4305bac1c682d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:35.114279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:35.114523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b11a7-9de9-4f5f-8ee5-3e8c79833406 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:35.116460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:35.117333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b11a7-9de9-4f5f-8ee5-3e8c79833406 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:35.119901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 930/1864] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:34 2022] DELETE /identity/v3/projects/db7a9603d2d74feeaa4305bac1c682d4 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:35.121488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:35.125453 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:35.196518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106b9e3-2d11-4c57-8d11-5edd195ba0c2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:35.201124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e200715-5f7e-4135-95f2-959de6b57f4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:35.202016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e200715-5f7e-4135-95f2-959de6b57f4a service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.202448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e200715-5f7e-4135-95f2-959de6b57f4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.202883 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e200715-5f7e-4135-95f2-959de6b57f4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.205237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8e200715-5f7e-4135-95f2-959de6b57f4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:35.228675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e200715-5f7e-4135-95f2-959de6b57f4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:35.230319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e200715-5f7e-4135-95f2-959de6b57f4a service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:35.240173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 935/1865] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:35 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:35.247820 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b11a7-9de9-4f5f-8ee5-3e8c79833406 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:35.248337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f84b11a7-9de9-4f5f-8ee5-3e8c79833406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:35.252274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 931/1866] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:35.255406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff5023-6cb6-427d-9377-68e641040d85 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.255804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff5023-6cb6-427d-9377-68e641040d85 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.256049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff5023-6cb6-427d-9377-68e641040d85 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.258139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 932/1867] 10.222.0.88 () {54 vars in 921 bytes} [Mon Jul 18 16:50:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:50:35.259235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff5023-6cb6-427d-9377-68e641040d85 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:35.261813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7aff5023-6cb6-427d-9377-68e641040d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:35.280170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b11a7-9de9-4f5f-8ee5-3e8c79833406 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:35.283561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff5023-6cb6-427d-9377-68e641040d85 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:35.286410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed954b-a317-4188-9b73-29be3f1233d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:35.287398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed954b-a317-4188-9b73-29be3f1233d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.287767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed954b-a317-4188-9b73-29be3f1233d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.288164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed954b-a317-4188-9b73-29be3f1233d7 admin admin] PATH_INFO: `/v3/users/7474fdb420124b3a9651d16c681138b3` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.289088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd2e6-1230-4e51-8b40-95c8c26e0e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:35.289826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed954b-a317-4188-9b73-29be3f1233d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7474fdb420124b3a9651d16c681138b3)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:35.290142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd2e6-1230-4e51-8b40-95c8c26e0e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.290592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd2e6-1230-4e51-8b40-95c8c26e0e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.291024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd2e6-1230-4e51-8b40-95c8c26e0e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.291943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acd2e6-1230-4e51-8b40-95c8c26e0e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:35.293544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed954b-a317-4188-9b73-29be3f1233d7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:35.294225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acd2e6-1230-4e51-8b40-95c8c26e0e9e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:35.309328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 933/1868] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:35.310879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d00fa-41d0-4c27-ac89-ce1028f78d60 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.311183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d00fa-41d0-4c27-ac89-ce1028f78d60 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.311541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d00fa-41d0-4c27-ac89-ce1028f78d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.338619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3b8d00fa-41d0-4c27-ac89-ce1028f78d60 None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:35.339866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3b8d00fa-41d0-4c27-ac89-ce1028f78d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:35.419205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 934/1869] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Jul 18 16:50:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:35.420170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d00fa-41d0-4c27-ac89-ce1028f78d60 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:35.433557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d00fa-41d0-4c27-ac89-ce1028f78d60 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:35.439105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a08c9-3a6e-43b7-95af-8c421fa1dd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:35.440077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a08c9-3a6e-43b7-95af-8c421fa1dd0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.440665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a08c9-3a6e-43b7-95af-8c421fa1dd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.441015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a08c9-3a6e-43b7-95af-8c421fa1dd0f admin admin] PATH_INFO: `/v3/projects/a1207a3917fa48e98b5afada1afda468/users/10d2c3c2ba1243978cbccec6174a51e4/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.460595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a08c9-3a6e-43b7-95af-8c421fa1dd0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1207a3917fa48e98b5afada1afda468, user_id=10d2c3c2ba1243978cbccec6174a51e4, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:35.463921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a08c9-3a6e-43b7-95af-8c421fa1dd0f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:35.518944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-34ed954b-a317-4188-9b73-29be3f1233d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7474fdb420124b3a9651d16c681138b3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:35.541307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-34ed954b-a317-4188-9b73-29be3f1233d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7474fdb420124b3a9651d16c681138b3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:35.563458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 935/1870] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:35 2022] PUT /identity/v3/projects/a1207a3917fa48e98b5afada1afda468/users/10d2c3c2ba1243978cbccec6174a51e4/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:35.564912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98f953-baeb-44f7-9944-e89d792986e9 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.565237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98f953-baeb-44f7-9944-e89d792986e9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.565522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98f953-baeb-44f7-9944-e89d792986e9 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.568869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 936/1871] 10.222.0.88 () {54 vars in 921 bytes} [Mon Jul 18 16:50:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:50:35.570905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98f953-baeb-44f7-9944-e89d792986e9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:35.575231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9e98f953-baeb-44f7-9944-e89d792986e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:35.585152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-34ed954b-a317-4188-9b73-29be3f1233d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7474fdb420124b3a9651d16c681138b3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:35.662178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98f953-baeb-44f7-9944-e89d792986e9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:35.668082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfd910-453e-4591-bc89-a0aedfb55823 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:35.668804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfd910-453e-4591-bc89-a0aedfb55823 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.669098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfd910-453e-4591-bc89-a0aedfb55823 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.669375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfd910-453e-4591-bc89-a0aedfb55823 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.680834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfd910-453e-4591-bc89-a0aedfb55823 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:35.682749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfd910-453e-4591-bc89-a0aedfb55823 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:35.695840 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 937/1872] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:35.697028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfd910-453e-4591-bc89-a0aedfb55823 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:35.712742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-34ed954b-a317-4188-9b73-29be3f1233d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7474fdb420124b3a9651d16c681138b3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:35.717513 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-34ed954b-a317-4188-9b73-29be3f1233d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7474fdb420124b3a9651d16c681138b3'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:35.728917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 936/1873] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:35 2022] DELETE /identity/v3/users/7474fdb420124b3a9651d16c681138b3 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:35.732399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa0304-58eb-4d1e-8288-0680a01c3eb4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.733266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa0304-58eb-4d1e-8288-0680a01c3eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.733787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa0304-58eb-4d1e-8288-0680a01c3eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.761342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-01aa0304-58eb-4d1e-8288-0680a01c3eb4 None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:35.764783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-01aa0304-58eb-4d1e-8288-0680a01c3eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:35.777593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfd910-453e-4591-bc89-a0aedfb55823 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:35.795973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:35.848447 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46ffee-728e-4187-8057-13c475e60c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:35.849215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46ffee-728e-4187-8057-13c475e60c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.849508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46ffee-728e-4187-8057-13c475e60c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.849795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46ffee-728e-4187-8057-13c475e60c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.850749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46ffee-728e-4187-8057-13c475e60c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:35.852433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46ffee-728e-4187-8057-13c475e60c7f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:35.869024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 938/1874] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:35.870385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46ffee-728e-4187-8057-13c475e60c7f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:35.872487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ba46ffee-728e-4187-8057-13c475e60c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:35.891682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46ffee-728e-4187-8057-13c475e60c7f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:35.910581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382dd4e-95db-4485-801e-7d58e85614d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:35.911921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382dd4e-95db-4485-801e-7d58e85614d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.912427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382dd4e-95db-4485-801e-7d58e85614d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.913427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382dd4e-95db-4485-801e-7d58e85614d9 admin admin] PATH_INFO: `/v3/projects/a1207a3917fa48e98b5afada1afda468/users/10d2c3c2ba1243978cbccec6174a51e4/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.919565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382dd4e-95db-4485-801e-7d58e85614d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1207a3917fa48e98b5afada1afda468, user_id=10d2c3c2ba1243978cbccec6174a51e4, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:35.921033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382dd4e-95db-4485-801e-7d58e85614d9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:35.937742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 939/1875] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:35 2022] PUT /identity/v3/projects/a1207a3917fa48e98b5afada1afda468/users/10d2c3c2ba1243978cbccec6174a51e4/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:35.947567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 937/1876] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Jul 18 16:50:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:35.952992 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f6a93-c2a3-4d75-917b-1a89b68ea261 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:35.954079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f6a93-c2a3-4d75-917b-1a89b68ea261 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:35.954527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f6a93-c2a3-4d75-917b-1a89b68ea261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:35.969367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa0304-58eb-4d1e-8288-0680a01c3eb4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:35.974045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-01aa0304-58eb-4d1e-8288-0680a01c3eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:36.000912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8d4f6a93-c2a3-4d75-917b-1a89b68ea261 None None] MFA Rules not processed for user `10d2c3c2ba1243978cbccec6174a51e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:36.002509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8d4f6a93-c2a3-4d75-917b-1a89b68ea261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:36.007067 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa0304-58eb-4d1e-8288-0680a01c3eb4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:36.112997 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 940/1877] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:35 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:36.120821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f6a93-c2a3-4d75-917b-1a89b68ea261 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:36.122565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8d4f6a93-c2a3-4d75-917b-1a89b68ea261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:36.183051 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29aeee-4470-4ab0-96e5-3cacc2b4c5eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:36.183798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29aeee-4470-4ab0-96e5-3cacc2b4c5eb service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:36.184188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29aeee-4470-4ab0-96e5-3cacc2b4c5eb service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:36.185696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29aeee-4470-4ab0-96e5-3cacc2b4c5eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:36.191265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f6a93-c2a3-4d75-917b-1a89b68ea261 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:36.195182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c67417-994f-4c98-8cd3-80ed7846225a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:36.195853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c67417-994f-4c98-8cd3-80ed7846225a service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:36.196153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c67417-994f-4c98-8cd3-80ed7846225a service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:36.196421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c67417-994f-4c98-8cd3-80ed7846225a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:36.198900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29aeee-4470-4ab0-96e5-3cacc2b4c5eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:36.209119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c67417-994f-4c98-8cd3-80ed7846225a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:36.209682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29aeee-4470-4ab0-96e5-3cacc2b4c5eb service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:36.211393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c67417-994f-4c98-8cd3-80ed7846225a service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:36.225087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 938/1878] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:36.231291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 941/1879] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:36.248208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29aeee-4470-4ab0-96e5-3cacc2b4c5eb service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:36.250102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cc29aeee-4470-4ab0-96e5-3cacc2b4c5eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:36.481941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29aeee-4470-4ab0-96e5-3cacc2b4c5eb service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:36.489180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:36.491308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:36.491779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:36.492212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] PATH_INFO: `/v3/projects/68ec43e2be644aadbbeaab87220a993c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:36.495537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=68ec43e2be644aadbbeaab87220a993c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:36.499220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:36.526887 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:36.534651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c67417-994f-4c98-8cd3-80ed7846225a service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:36.545855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c67417-994f-4c98-8cd3-80ed7846225a service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:36.550407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fce5c7-275b-4e5e-ae11-d75eb07ed845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:36.551863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fce5c7-275b-4e5e-ae11-d75eb07ed845 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:36.552294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fce5c7-275b-4e5e-ae11-d75eb07ed845 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:36.552938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fce5c7-275b-4e5e-ae11-d75eb07ed845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:36.554988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-07fce5c7-275b-4e5e-ae11-d75eb07ed845 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:36.645290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fce5c7-275b-4e5e-ae11-d75eb07ed845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:36.647640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fce5c7-275b-4e5e-ae11-d75eb07ed845 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:36.683547 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ec43e2be644aadbbeaab87220a993c', 'request_id': 'req-c5327df5-0c23-42bc-aece-2872ff15a4fc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:36.687109 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 942/1880] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:36.713775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ec43e2be644aadbbeaab87220a993c', 'request_id': 'req-c5327df5-0c23-42bc-aece-2872ff15a4fc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:36.736596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ec43e2be644aadbbeaab87220a993c', 'request_id': 'req-c5327df5-0c23-42bc-aece-2872ff15a4fc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:36.852351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] The token cache is being invalidate because project 68ec43e2be644aadbbeaab87220a993c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:36.852594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:36.860243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:36.866488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 939/1881] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:36 2022] DELETE /identity/v3/projects/68ec43e2be644aadbbeaab87220a993c => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:36.871140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fce5c7-275b-4e5e-ae11-d75eb07ed845 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:36.893814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:36.909943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fce5c7-275b-4e5e-ae11-d75eb07ed845 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:37.032586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b8ade-76a3-488c-8e5e-ef6b54e32b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:37.034292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b8ade-76a3-488c-8e5e-ef6b54e32b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:37.034808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b8ade-76a3-488c-8e5e-ef6b54e32b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:37.035655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b8ade-76a3-488c-8e5e-ef6b54e32b78 admin admin] PATH_INFO: `/v3/users/d71453803e284c929ceb65456809db13` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:37.037544 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b8ade-76a3-488c-8e5e-ef6b54e32b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d71453803e284c929ceb65456809db13)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:37.040265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b8ade-76a3-488c-8e5e-ef6b54e32b78 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:37.233608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:37.235368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:37.281090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5327df5-0c23-42bc-aece-2872ff15a4fc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:37.326683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a04b8ade-76a3-488c-8e5e-ef6b54e32b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd71453803e284c929ceb65456809db13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:37.348513 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:37.353512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:37.353909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:37.354161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] PATH_INFO: `/v3/projects/dbd35d396f1147b396ac0f8e6b493714` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:37.362153 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd35d396f1147b396ac0f8e6b493714)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:37.364402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:37.454807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a04b8ade-76a3-488c-8e5e-ef6b54e32b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd71453803e284c929ceb65456809db13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:37.513551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a04b8ade-76a3-488c-8e5e-ef6b54e32b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd71453803e284c929ceb65456809db13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:37.607766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a04b8ade-76a3-488c-8e5e-ef6b54e32b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd71453803e284c929ceb65456809db13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:37.608724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a04b8ade-76a3-488c-8e5e-ef6b54e32b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd71453803e284c929ceb65456809db13'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:37.637824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 943/1882] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:36 2022] DELETE /identity/v3/users/d71453803e284c929ceb65456809db13 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:37.665280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd35d396f1147b396ac0f8e6b493714', 'request_id': 'req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:37.684206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd35d396f1147b396ac0f8e6b493714', 'request_id': 'req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:37.735765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd35d396f1147b396ac0f8e6b493714', 'request_id': 'req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:37.853609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] The token cache is being invalidate because project dbd35d396f1147b396ac0f8e6b493714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:37.854900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:37.856735 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1bf9d3e8-0716-49b4-b895-39b0d8afc9c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:37.865225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 940/1883] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:37 2022] DELETE /identity/v3/projects/dbd35d396f1147b396ac0f8e6b493714 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:37.875142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5711d45-562e-4ae7-850a-f5db7d19b35c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:37.875142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5711d45-562e-4ae7-850a-f5db7d19b35c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:37.875142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5711d45-562e-4ae7-850a-f5db7d19b35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:37.941041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f5711d45-562e-4ae7-850a-f5db7d19b35c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:37.942743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f5711d45-562e-4ae7-850a-f5db7d19b35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:38.100003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 944/1884] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:38.111299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef37e4-6f13-4c7c-a3bd-6d02f049587b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:38.112156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef37e4-6f13-4c7c-a3bd-6d02f049587b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:38.113870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef37e4-6f13-4c7c-a3bd-6d02f049587b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:38.220854 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-51ef37e4-6f13-4c7c-a3bd-6d02f049587b None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:38.223090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-51ef37e4-6f13-4c7c-a3bd-6d02f049587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:38.280871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:38.285767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 941/1885] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:38.296850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5711d45-562e-4ae7-850a-f5db7d19b35c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:38.310877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5711d45-562e-4ae7-850a-f5db7d19b35c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:38.319253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0af59-e048-4f26-8a97-92eac227aa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:38.320203 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0af59-e048-4f26-8a97-92eac227aa0d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:38.320556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0af59-e048-4f26-8a97-92eac227aa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:38.320989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0af59-e048-4f26-8a97-92eac227aa0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:38.321853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0af59-e048-4f26-8a97-92eac227aa0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:38.322232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0af59-e048-4f26-8a97-92eac227aa0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:38.325281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0af59-e048-4f26-8a97-92eac227aa0d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:38.384461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 945/1886] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:38.386037 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef37e4-6f13-4c7c-a3bd-6d02f049587b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:38.401765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef37e4-6f13-4c7c-a3bd-6d02f049587b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:38.415653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2ba42-7016-409c-8ca3-1c18f9b46ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:38.418090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ba42-7016-409c-8ca3-1c18f9b46ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:38.418090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ba42-7016-409c-8ca3-1c18f9b46ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:38.418090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ba42-7016-409c-8ca3-1c18f9b46ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:38.420157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2ba42-7016-409c-8ca3-1c18f9b46ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:38.427509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2ba42-7016-409c-8ca3-1c18f9b46ae5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:38.638354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:39.141273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 942/1887] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:38 2022] POST /identity/v3/projects => generated 381 bytes in 756 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:39.145357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0af59-e048-4f26-8a97-92eac227aa0d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:39.166784 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0af59-e048-4f26-8a97-92eac227aa0d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:39.342994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b8bc7-dd6d-408d-bfc6-eb2a8c6cda20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:39.343664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8bc7-dd6d-408d-bfc6-eb2a8c6cda20 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:39.343991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8bc7-dd6d-408d-bfc6-eb2a8c6cda20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:39.344270 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b8bc7-dd6d-408d-bfc6-eb2a8c6cda20 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:39.351514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b8bc7-dd6d-408d-bfc6-eb2a8c6cda20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:39.351514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b8bc7-dd6d-408d-bfc6-eb2a8c6cda20 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:39.547689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 946/1888] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:39 2022] POST /identity/v3/users => generated 291 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:50:39.549221 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2ba42-7016-409c-8ca3-1c18f9b46ae5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:39.594674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2ba42-7016-409c-8ca3-1c18f9b46ae5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:39.618762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7e948-2e75-4ccb-bf0e-96b0fd6958b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:39.631087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7e948-2e75-4ccb-bf0e-96b0fd6958b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:39.634251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7e948-2e75-4ccb-bf0e-96b0fd6958b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:39.635205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7e948-2e75-4ccb-bf0e-96b0fd6958b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:39.645860 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7e948-2e75-4ccb-bf0e-96b0fd6958b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:39.654671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7e948-2e75-4ccb-bf0e-96b0fd6958b6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:39.676302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 943/1889] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:39.678944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b8bc7-dd6d-408d-bfc6-eb2a8c6cda20 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:39.693647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b8bc7-dd6d-408d-bfc6-eb2a8c6cda20 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:39.703160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a3b23-8c8a-4e21-b89d-75bcfb989ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:39.706332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a3b23-8c8a-4e21-b89d-75bcfb989ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:39.706721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a3b23-8c8a-4e21-b89d-75bcfb989ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:39.707117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a3b23-8c8a-4e21-b89d-75bcfb989ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:39.707943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a3b23-8c8a-4e21-b89d-75bcfb989ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:39.712467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a3b23-8c8a-4e21-b89d-75bcfb989ecc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:39.740494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 947/1890] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:39.745422 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7e948-2e75-4ccb-bf0e-96b0fd6958b6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:39.768966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7e948-2e75-4ccb-bf0e-96b0fd6958b6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:39.780004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d018d45-36c7-4ce1-b0f1-177202cd0e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:39.784211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d018d45-36c7-4ce1-b0f1-177202cd0e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:39.786236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d018d45-36c7-4ce1-b0f1-177202cd0e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:39.786849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d018d45-36c7-4ce1-b0f1-177202cd0e5f admin admin] PATH_INFO: `/v3/projects/61a80f5fe0794f679d4b5b0c7a4593ce/users/57f58ee106cf4dcb9014f6c118b9c7c4/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:39.815449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d018d45-36c7-4ce1-b0f1-177202cd0e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a80f5fe0794f679d4b5b0c7a4593ce, user_id=57f58ee106cf4dcb9014f6c118b9c7c4, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:39.823177 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d018d45-36c7-4ce1-b0f1-177202cd0e5f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:39.980872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 944/1891] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:39 2022] PUT /identity/v3/projects/61a80f5fe0794f679d4b5b0c7a4593ce/users/57f58ee106cf4dcb9014f6c118b9c7c4/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:39.993132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ff53b-668e-4eb9-9883-f94ee2166b67 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:39.994080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ff53b-668e-4eb9-9883-f94ee2166b67 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:39.994502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ff53b-668e-4eb9-9883-f94ee2166b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:40.042584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-313ff53b-668e-4eb9-9883-f94ee2166b67 None None] MFA Rules not processed for user `57f58ee106cf4dcb9014f6c118b9c7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:40.045314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-313ff53b-668e-4eb9-9883-f94ee2166b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:40.314824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 948/1892] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:40.319482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d018d45-36c7-4ce1-b0f1-177202cd0e5f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:40.601907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d018d45-36c7-4ce1-b0f1-177202cd0e5f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:40.619342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed44c3-4457-4850-9611-e0c2286218fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:40.620687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed44c3-4457-4850-9611-e0c2286218fc admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:40.621176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed44c3-4457-4850-9611-e0c2286218fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:40.621664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed44c3-4457-4850-9611-e0c2286218fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:40.622569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed44c3-4457-4850-9611-e0c2286218fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:40.633650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed44c3-4457-4850-9611-e0c2286218fc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:40.793300 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 945/1893] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:40 2022] POST /identity/v3/projects => generated 383 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:40.794522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ff53b-668e-4eb9-9883-f94ee2166b67 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:40.820317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ff53b-668e-4eb9-9883-f94ee2166b67 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:40.877611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed44c3-4457-4850-9611-e0c2286218fc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:40.880102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-92ed44c3-4457-4850-9611-e0c2286218fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:40.929480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:40.932441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ced43-94ea-4052-ae96-5844d4ea94bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:40.933464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ced43-94ea-4052-ae96-5844d4ea94bb admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:40.933981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ced43-94ea-4052-ae96-5844d4ea94bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:40.934794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ced43-94ea-4052-ae96-5844d4ea94bb admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:40.935729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ced43-94ea-4052-ae96-5844d4ea94bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:40.938789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ced43-94ea-4052-ae96-5844d4ea94bb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:41.024869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed44c3-4457-4850-9611-e0c2286218fc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:41.045561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:41.046549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.046911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.047268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] PATH_INFO: `/v3/projects/37e0a6554bf04c6eaa4008a4532f299f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.048730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37e0a6554bf04c6eaa4008a4532f299f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:41.051155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:41.051568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 949/1894] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:40 2022] POST /identity/v3/users => generated 298 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:41.055150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ced43-94ea-4052-ae96-5844d4ea94bb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:41.075152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ced43-94ea-4052-ae96-5844d4ea94bb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:41.088090 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade0af1-2dec-416b-9ed6-f47af5b7f7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:41.089190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade0af1-2dec-416b-9ed6-f47af5b7f7fd admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.089624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade0af1-2dec-416b-9ed6-f47af5b7f7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.090839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade0af1-2dec-416b-9ed6-f47af5b7f7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.092455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade0af1-2dec-416b-9ed6-f47af5b7f7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:41.094310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade0af1-2dec-416b-9ed6-f47af5b7f7fd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:41.115876 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 950/1895] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:41.117104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade0af1-2dec-416b-9ed6-f47af5b7f7fd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:41.142770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade0af1-2dec-416b-9ed6-f47af5b7f7fd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:41.146555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbb696-ed68-4c05-b3c8-8c007db4e344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:41.147756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbb696-ed68-4c05-b3c8-8c007db4e344 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.148205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbb696-ed68-4c05-b3c8-8c007db4e344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.148694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbb696-ed68-4c05-b3c8-8c007db4e344 admin admin] PATH_INFO: `/v3/projects/5f1834b64f0f4341a9fb02c8534d54db/users/6066f0c6dbad4921b84e0cc4d32e629a/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.184548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbb696-ed68-4c05-b3c8-8c007db4e344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1834b64f0f4341a9fb02c8534d54db, user_id=6066f0c6dbad4921b84e0cc4d32e629a, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:41.187961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbb696-ed68-4c05-b3c8-8c007db4e344 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:41.193636 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37e0a6554bf04c6eaa4008a4532f299f', 'request_id': 'req-171bffbd-fde3-4987-91fb-4b28b1e6efd6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:41.201055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37e0a6554bf04c6eaa4008a4532f299f', 'request_id': 'req-171bffbd-fde3-4987-91fb-4b28b1e6efd6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:41.209933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37e0a6554bf04c6eaa4008a4532f299f', 'request_id': 'req-171bffbd-fde3-4987-91fb-4b28b1e6efd6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:41.245235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 951/1896] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:41 2022] PUT /identity/v3/projects/5f1834b64f0f4341a9fb02c8534d54db/users/6066f0c6dbad4921b84e0cc4d32e629a/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:41.249422 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbb696-ed68-4c05-b3c8-8c007db4e344 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:41.256771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:41.285708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] The token cache is being invalidate because project 37e0a6554bf04c6eaa4008a4532f299f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:41.286459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:41.287864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-171bffbd-fde3-4987-91fb-4b28b1e6efd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:41.293725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 946/1897] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:40 2022] DELETE /identity/v3/projects/37e0a6554bf04c6eaa4008a4532f299f => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:41.295519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214583d9-001f-40bf-bd01-007e9aeab92f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.296010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214583d9-001f-40bf-bd01-007e9aeab92f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.296490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214583d9-001f-40bf-bd01-007e9aeab92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.322699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-214583d9-001f-40bf-bd01-007e9aeab92f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:41.325022 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-214583d9-001f-40bf-bd01-007e9aeab92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:41.461032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 947/1898] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:41.473387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130f680-75cc-45b6-8f4a-dfbc22bf0511 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.476032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130f680-75cc-45b6-8f4a-dfbc22bf0511 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.476724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130f680-75cc-45b6-8f4a-dfbc22bf0511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.478662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbb696-ed68-4c05-b3c8-8c007db4e344 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:41.489393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fd647-ab46-4c48-a115-d51e180839f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:41.490960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fd647-ab46-4c48-a115-d51e180839f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.491709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fd647-ab46-4c48-a115-d51e180839f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.493532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fd647-ab46-4c48-a115-d51e180839f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.494777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fd647-ab46-4c48-a115-d51e180839f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:41.496587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fd647-ab46-4c48-a115-d51e180839f9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:41.508410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 952/1899] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:41.509736 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b130f680-75cc-45b6-8f4a-dfbc22bf0511 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:41.511190 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b130f680-75cc-45b6-8f4a-dfbc22bf0511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:41.512300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fd647-ab46-4c48-a115-d51e180839f9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:41.517839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b06fd647-ab46-4c48-a115-d51e180839f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:41.525777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 948/1900] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:41.526939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130f680-75cc-45b6-8f4a-dfbc22bf0511 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:41.539452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130f680-75cc-45b6-8f4a-dfbc22bf0511 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:41.541361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fd647-ab46-4c48-a115-d51e180839f9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:41.545059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ed988-6c9f-4c8f-bef9-f25f3828aa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:41.545757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed988-6c9f-4c8f-bef9-f25f3828aa11 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.546244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed988-6c9f-4c8f-bef9-f25f3828aa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.546796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ed988-6c9f-4c8f-bef9-f25f3828aa11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.547541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ed988-6c9f-4c8f-bef9-f25f3828aa11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:41.547756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492433a-800f-4c0b-a87c-edd445efe084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:41.547869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ed988-6c9f-4c8f-bef9-f25f3828aa11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:41.549094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492433a-800f-4c0b-a87c-edd445efe084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.549570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492433a-800f-4c0b-a87c-edd445efe084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.549816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ed988-6c9f-4c8f-bef9-f25f3828aa11 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:41.549980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492433a-800f-4c0b-a87c-edd445efe084 admin admin] PATH_INFO: `/v3/projects/5f1834b64f0f4341a9fb02c8534d54db/users/6066f0c6dbad4921b84e0cc4d32e629a/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.556970 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492433a-800f-4c0b-a87c-edd445efe084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1834b64f0f4341a9fb02c8534d54db, user_id=6066f0c6dbad4921b84e0cc4d32e629a, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:41.560108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492433a-800f-4c0b-a87c-edd445efe084 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:41.564088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 949/1901] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:41.569337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ed988-6c9f-4c8f-bef9-f25f3828aa11 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:41.575745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 953/1902] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:41 2022] PUT /identity/v3/projects/5f1834b64f0f4341a9fb02c8534d54db/users/6066f0c6dbad4921b84e0cc4d32e629a/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:41.580519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ed988-6c9f-4c8f-bef9-f25f3828aa11 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:41.588496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bbfb1-1573-42b6-8aec-203d55aeb6c0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.589091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bbfb1-1573-42b6-8aec-203d55aeb6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.589502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bbfb1-1573-42b6-8aec-203d55aeb6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.622075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-468bbfb1-1573-42b6-8aec-203d55aeb6c0 None None] MFA Rules not processed for user `6066f0c6dbad4921b84e0cc4d32e629a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:41.623494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-468bbfb1-1573-42b6-8aec-203d55aeb6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:41.646294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4aa37-c8f1-4deb-a250-02eaace2b831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:41.646961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4aa37-c8f1-4deb-a250-02eaace2b831 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.647248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4aa37-c8f1-4deb-a250-02eaace2b831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.647525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4aa37-c8f1-4deb-a250-02eaace2b831 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.648218 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4aa37-c8f1-4deb-a250-02eaace2b831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:41.651241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4aa37-c8f1-4deb-a250-02eaace2b831 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:41.703983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 954/1903] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:41.712233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bbfb1-1573-42b6-8aec-203d55aeb6c0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:41.714433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-468bbfb1-1573-42b6-8aec-203d55aeb6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:41.792546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bbfb1-1573-42b6-8aec-203d55aeb6c0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:41.796499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027e858-c851-40e7-8fa5-417cf772d12b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:41.797235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027e858-c851-40e7-8fa5-417cf772d12b service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.797540 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027e858-c851-40e7-8fa5-417cf772d12b service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.797827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027e858-c851-40e7-8fa5-417cf772d12b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.809099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027e858-c851-40e7-8fa5-417cf772d12b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:41.810575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027e858-c851-40e7-8fa5-417cf772d12b service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:41.826421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 950/1904] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:41 2022] POST /identity/v3/projects => generated 365 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:41.827654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4aa37-c8f1-4deb-a250-02eaace2b831 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:41.828645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 955/1905] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:41.917743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4aa37-c8f1-4deb-a250-02eaace2b831 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:41.922796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401575c-9262-4886-8a6f-cb3c1523787e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:41.923663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401575c-9262-4886-8a6f-cb3c1523787e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:41.924039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401575c-9262-4886-8a6f-cb3c1523787e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:41.924378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401575c-9262-4886-8a6f-cb3c1523787e admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:41.925237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401575c-9262-4886-8a6f-cb3c1523787e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:41.927876 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401575c-9262-4886-8a6f-cb3c1523787e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:41.951905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027e858-c851-40e7-8fa5-417cf772d12b service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:41.960770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027e858-c851-40e7-8fa5-417cf772d12b service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:41.967412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:42.014792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43351123-3618-454a-b8ea-41611bc27b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:42.015455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43351123-3618-454a-b8ea-41611bc27b13 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.015741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43351123-3618-454a-b8ea-41611bc27b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.016020 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43351123-3618-454a-b8ea-41611bc27b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.017829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-43351123-3618-454a-b8ea-41611bc27b13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:42.033397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 951/1906] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:41 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:42.034339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401575c-9262-4886-8a6f-cb3c1523787e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.045721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401575c-9262-4886-8a6f-cb3c1523787e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:42.050830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3416af-d39c-4749-bc55-3ba3b5f35752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:42.051501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3416af-d39c-4749-bc55-3ba3b5f35752 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.051787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3416af-d39c-4749-bc55-3ba3b5f35752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.052064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3416af-d39c-4749-bc55-3ba3b5f35752 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.052802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3416af-d39c-4749-bc55-3ba3b5f35752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:42.054624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3416af-d39c-4749-bc55-3ba3b5f35752 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:42.069411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 952/1907] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:42.070265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3416af-d39c-4749-bc55-3ba3b5f35752 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.079990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:42.082894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3416af-d39c-4749-bc55-3ba3b5f35752 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:42.088207 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b225eee-18ad-410f-823c-601879fa67c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:42.089020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b225eee-18ad-410f-823c-601879fa67c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.089313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b225eee-18ad-410f-823c-601879fa67c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.089591 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b225eee-18ad-410f-823c-601879fa67c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.090326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b225eee-18ad-410f-823c-601879fa67c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:42.091943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b225eee-18ad-410f-823c-601879fa67c2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:42.093559 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43351123-3618-454a-b8ea-41611bc27b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:42.095403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43351123-3618-454a-b8ea-41611bc27b13 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:42.103774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 953/1908] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:42.107986 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b225eee-18ad-410f-823c-601879fa67c2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.111964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 956/1909] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:42.118043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b225eee-18ad-410f-823c-601879fa67c2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:42.124445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e27b-81b0-4255-96a9-9c71d2a4684c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:42.125632 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e27b-81b0-4255-96a9-9c71d2a4684c admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.126114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e27b-81b0-4255-96a9-9c71d2a4684c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.126496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e27b-81b0-4255-96a9-9c71d2a4684c admin admin] PATH_INFO: `/v3/projects/3e2afa21099145bcbb0f76e34dde60c2/users/c08be6237d074e2ca37ad2db5653ca0d/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.146196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1e27b-81b0-4255-96a9-9c71d2a4684c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2afa21099145bcbb0f76e34dde60c2, user_id=c08be6237d074e2ca37ad2db5653ca0d, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:42.148865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1e27b-81b0-4255-96a9-9c71d2a4684c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:42.246576 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 954/1910] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:42 2022] PUT /identity/v3/projects/3e2afa21099145bcbb0f76e34dde60c2/users/c08be6237d074e2ca37ad2db5653ca0d/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:42.258000 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe295e8-c570-45c0-893f-f7589943f642 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.258508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe295e8-c570-45c0-893f-f7589943f642 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.263547 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe295e8-c570-45c0-893f-f7589943f642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.289391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-abe295e8-c570-45c0-893f-f7589943f642 None None] MFA Rules not processed for user `c08be6237d074e2ca37ad2db5653ca0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:42.290693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-abe295e8-c570-45c0-893f-f7589943f642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:42.367285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 957/1911] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:42.368583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e27b-81b0-4255-96a9-9c71d2a4684c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.432991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e27b-81b0-4255-96a9-9c71d2a4684c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:42.438040 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206d0d2-126e-4981-923c-760574681a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:42.438705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d0d2-126e-4981-923c-760574681a62 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.439018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d0d2-126e-4981-923c-760574681a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.439299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d0d2-126e-4981-923c-760574681a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.440000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206d0d2-126e-4981-923c-760574681a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:42.442009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206d0d2-126e-4981-923c-760574681a62 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:42.526774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:42.534536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 955/1912] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:42 2022] POST /identity/v3/projects => generated 363 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:42.539521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe295e8-c570-45c0-893f-f7589943f642 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.551450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe295e8-c570-45c0-893f-f7589943f642 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:42.623940 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba76da6-4a4b-4d1d-8845-fbcb50de1077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:42.624659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba76da6-4a4b-4d1d-8845-fbcb50de1077 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.624981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba76da6-4a4b-4d1d-8845-fbcb50de1077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.625449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba76da6-4a4b-4d1d-8845-fbcb50de1077 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.626328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba76da6-4a4b-4d1d-8845-fbcb50de1077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:42.630588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba76da6-4a4b-4d1d-8845-fbcb50de1077 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:42.708801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 958/1913] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:42 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:42.715575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206d0d2-126e-4981-923c-760574681a62 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.726332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206d0d2-126e-4981-923c-760574681a62 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:42.733705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b7d30-75b3-4651-9cbd-1ab673f5d268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:42.734616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b7d30-75b3-4651-9cbd-1ab673f5d268 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.735106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b7d30-75b3-4651-9cbd-1ab673f5d268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.735521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b7d30-75b3-4651-9cbd-1ab673f5d268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.736288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b7d30-75b3-4651-9cbd-1ab673f5d268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:42.739238 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b7d30-75b3-4651-9cbd-1ab673f5d268 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:42.756639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba76da6-4a4b-4d1d-8845-fbcb50de1077 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.757238 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 956/1914] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:42.768383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba76da6-4a4b-4d1d-8845-fbcb50de1077 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:42.774549 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f0b34-f62c-432b-bebb-64025f8121b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:42.775465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0b34-f62c-432b-bebb-64025f8121b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.775748 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0b34-f62c-432b-bebb-64025f8121b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.776024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f0b34-f62c-432b-bebb-64025f8121b9 admin admin] PATH_INFO: `/v3/projects/0cfd92c847054ab9a903ead719d033f7/users/82f71e5a1dea4fed9a9f5ad9ce51b09f/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.797155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f0b34-f62c-432b-bebb-64025f8121b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cfd92c847054ab9a903ead719d033f7, user_id=82f71e5a1dea4fed9a9f5ad9ce51b09f, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:42.800124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f0b34-f62c-432b-bebb-64025f8121b9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:42.875140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 959/1915] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:42 2022] PUT /identity/v3/projects/0cfd92c847054ab9a903ead719d033f7/users/82f71e5a1dea4fed9a9f5ad9ce51b09f/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:42.879179 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b7d30-75b3-4651-9cbd-1ab673f5d268 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.890194 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b7d30-75b3-4651-9cbd-1ab673f5d268 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:42.965658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c9d3b-f848-404f-8dff-6bc241aac323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:42.966344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c9d3b-f848-404f-8dff-6bc241aac323 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:42.966691 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c9d3b-f848-404f-8dff-6bc241aac323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:42.966909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c9d3b-f848-404f-8dff-6bc241aac323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:42.967650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c9d3b-f848-404f-8dff-6bc241aac323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:42.969547 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c9d3b-f848-404f-8dff-6bc241aac323 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:42.977408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:42.985646 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 957/1916] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:42.986785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f0b34-f62c-432b-bebb-64025f8121b9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:42.997726 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f0b34-f62c-432b-bebb-64025f8121b9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:43.002482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cb88d-f4d5-471d-a4e5-44bf8da9b9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:43.003394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb88d-f4d5-471d-a4e5-44bf8da9b9fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:43.003707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb88d-f4d5-471d-a4e5-44bf8da9b9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:43.004045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb88d-f4d5-471d-a4e5-44bf8da9b9fb admin admin] PATH_INFO: `/v3/projects/0cfd92c847054ab9a903ead719d033f7/users/82f71e5a1dea4fed9a9f5ad9ce51b09f/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:43.006065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cb88d-f4d5-471d-a4e5-44bf8da9b9fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cfd92c847054ab9a903ead719d033f7, user_id=82f71e5a1dea4fed9a9f5ad9ce51b09f, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:43.016509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cb88d-f4d5-471d-a4e5-44bf8da9b9fb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:43.025611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 960/1917] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:42 2022] PUT /identity/v3/projects/0cfd92c847054ab9a903ead719d033f7/users/82f71e5a1dea4fed9a9f5ad9ce51b09f/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:43.036411 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b34a-c082-442e-9b5b-ae9217492ec2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:43.036706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b34a-c082-442e-9b5b-ae9217492ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:43.037018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b34a-c082-442e-9b5b-ae9217492ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:43.065067 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e8e7b34a-c082-442e-9b5b-ae9217492ec2 None None] MFA Rules not processed for user `82f71e5a1dea4fed9a9f5ad9ce51b09f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:43.066407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e8e7b34a-c082-442e-9b5b-ae9217492ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:43.134575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 958/1918] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:43 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:43.146684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cb88d-f4d5-471d-a4e5-44bf8da9b9fb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:43.204293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cb88d-f4d5-471d-a4e5-44bf8da9b9fb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:43.208494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c9cec-180f-4e84-a138-c26dd105571d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:43.209167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c9cec-180f-4e84-a138-c26dd105571d service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:43.209668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c9cec-180f-4e84-a138-c26dd105571d service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:43.210118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c9cec-180f-4e84-a138-c26dd105571d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:43.220917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c9cec-180f-4e84-a138-c26dd105571d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:43.221860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c9cec-180f-4e84-a138-c26dd105571d service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:43.231890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 961/1919] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:43 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:43.447524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7b34a-c082-442e-9b5b-ae9217492ec2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:43.458349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7b34a-c082-442e-9b5b-ae9217492ec2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:43.462066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd7c27-37bd-46ef-bd6d-d1ac237cc787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:43.462715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd7c27-37bd-46ef-bd6d-d1ac237cc787 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:43.463258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd7c27-37bd-46ef-bd6d-d1ac237cc787 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:43.463533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd7c27-37bd-46ef-bd6d-d1ac237cc787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:43.521754 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd7c27-37bd-46ef-bd6d-d1ac237cc787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:43.523283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd7c27-37bd-46ef-bd6d-d1ac237cc787 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:43.525258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:43.532721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 959/1920] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:44.073041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:44.703313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:45.633197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:45.799730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:50.257534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c9cec-180f-4e84-a138-c26dd105571d service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:50.279161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:50.335118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c9cec-180f-4e84-a138-c26dd105571d service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:50.340121 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:50.341041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.341357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.341670 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] PATH_INFO: `/v3/users/c08be6237d074e2ca37ad2db5653ca0d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.343107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08be6237d074e2ca37ad2db5653ca0d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:50.345108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:50.591170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08be6237d074e2ca37ad2db5653ca0d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:50.604382 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08be6237d074e2ca37ad2db5653ca0d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:50.611456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08be6237d074e2ca37ad2db5653ca0d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:50.678854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08be6237d074e2ca37ad2db5653ca0d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:50.683143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08be6237d074e2ca37ad2db5653ca0d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:50.685880 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 962/1921] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:50 2022] DELETE /identity/v3/users/c08be6237d074e2ca37ad2db5653ca0d => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:50.690487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073b3f0-11c3-4a96-b1d2-7e065c9dc327 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.691502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073b3f0-11c3-4a96-b1d2-7e065c9dc327 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.691692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073b3f0-11c3-4a96-b1d2-7e065c9dc327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.709618 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1073b3f0-11c3-4a96-b1d2-7e065c9dc327 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:50.710858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1073b3f0-11c3-4a96-b1d2-7e065c9dc327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:50.758396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:50.770220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 960/1922] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:50.780431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:50.781710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:50.841376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567d9b6-a9de-4295-9dc0-775b3a16c5ba admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:50.845046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce9eda-a3c2-4cd1-aaa5-80003d20f5b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:50.845698 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce9eda-a3c2-4cd1-aaa5-80003d20f5b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:50.845987 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce9eda-a3c2-4cd1-aaa5-80003d20f5b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:50.846267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce9eda-a3c2-4cd1-aaa5-80003d20f5b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:50.855356 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce9eda-a3c2-4cd1-aaa5-80003d20f5b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:50.856790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce9eda-a3c2-4cd1-aaa5-80003d20f5b1 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:50.866963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 963/1923] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:51.282046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073b3f0-11c3-4a96-b1d2-7e065c9dc327 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:51.283729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1073b3f0-11c3-4a96-b1d2-7e065c9dc327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:51.296183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073b3f0-11c3-4a96-b1d2-7e065c9dc327 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:51.301776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:51.308183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:51.308183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:51.308183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] PATH_INFO: `/v3/projects/3e2afa21099145bcbb0f76e34dde60c2` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:51.308183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e2afa21099145bcbb0f76e34dde60c2)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:51.308183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:51.455991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e2afa21099145bcbb0f76e34dde60c2', 'request_id': 'req-78077b81-9e08-4b46-9135-1d7bb72001de'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.462316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e2afa21099145bcbb0f76e34dde60c2', 'request_id': 'req-78077b81-9e08-4b46-9135-1d7bb72001de'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.468558 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e2afa21099145bcbb0f76e34dde60c2', 'request_id': 'req-78077b81-9e08-4b46-9135-1d7bb72001de'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.541640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] The token cache is being invalidate because project 3e2afa21099145bcbb0f76e34dde60c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:51.542059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.542710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.544293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 961/1924] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:51 2022] DELETE /identity/v3/projects/3e2afa21099145bcbb0f76e34dde60c2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:51.548276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce9eda-a3c2-4cd1-aaa5-80003d20f5b1 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:51.558977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce9eda-a3c2-4cd1-aaa5-80003d20f5b1 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:51.600480 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:51.618634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8121d-cd0b-47ef-bb7c-e2bf3af9adbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:51.619543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8121d-cd0b-47ef-bb7c-e2bf3af9adbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:51.619821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8121d-cd0b-47ef-bb7c-e2bf3af9adbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:51.620134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8121d-cd0b-47ef-bb7c-e2bf3af9adbc admin admin] PATH_INFO: `/v3/users/82f71e5a1dea4fed9a9f5ad9ce51b09f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:51.621361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8121d-cd0b-47ef-bb7c-e2bf3af9adbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f71e5a1dea4fed9a9f5ad9ce51b09f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:51.623353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8121d-cd0b-47ef-bb7c-e2bf3af9adbc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:51.811961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81e8121d-cd0b-47ef-bb7c-e2bf3af9adbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f71e5a1dea4fed9a9f5ad9ce51b09f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.824065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81e8121d-cd0b-47ef-bb7c-e2bf3af9adbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f71e5a1dea4fed9a9f5ad9ce51b09f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.830926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81e8121d-cd0b-47ef-bb7c-e2bf3af9adbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f71e5a1dea4fed9a9f5ad9ce51b09f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.894668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81e8121d-cd0b-47ef-bb7c-e2bf3af9adbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f71e5a1dea4fed9a9f5ad9ce51b09f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.895376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81e8121d-cd0b-47ef-bb7c-e2bf3af9adbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f71e5a1dea4fed9a9f5ad9ce51b09f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:51.902646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 964/1925] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:51 2022] DELETE /identity/v3/users/82f71e5a1dea4fed9a9f5ad9ce51b09f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:52.337027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.338923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:52.352226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78077b81-9e08-4b46-9135-1d7bb72001de admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:52.422056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:52.423209 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.423554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.424030 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] PATH_INFO: `/v3/projects/0cfd92c847054ab9a903ead719d033f7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.426968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cfd92c847054ab9a903ead719d033f7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:52.430258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:52.532434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cfd92c847054ab9a903ead719d033f7', 'request_id': 'req-580542d9-610c-48ab-bc20-cd74a7592516'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:52.539697 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cfd92c847054ab9a903ead719d033f7', 'request_id': 'req-580542d9-610c-48ab-bc20-cd74a7592516'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:52.546283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cfd92c847054ab9a903ead719d033f7', 'request_id': 'req-580542d9-610c-48ab-bc20-cd74a7592516'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:52.650470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] The token cache is being invalidate because project 0cfd92c847054ab9a903ead719d033f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:52.650704 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:52.651330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580542d9-610c-48ab-bc20-cd74a7592516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:52.652927 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 962/1926] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:52 2022] DELETE /identity/v3/projects/0cfd92c847054ab9a903ead719d033f7 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:52.658296 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ec1b8-9601-4d83-87de-dcd4a847afbb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.658761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ec1b8-9601-4d83-87de-dcd4a847afbb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.659257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ec1b8-9601-4d83-87de-dcd4a847afbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.678146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-580ec1b8-9601-4d83-87de-dcd4a847afbb None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:52.689255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-580ec1b8-9601-4d83-87de-dcd4a847afbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:52.740777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 965/1927] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:52.748804 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9fe93-7486-4e4b-8c03-659a138ad7f5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.749264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9fe93-7486-4e4b-8c03-659a138ad7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.749591 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9fe93-7486-4e4b-8c03-659a138ad7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.768934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-77a9fe93-7486-4e4b-8c03-659a138ad7f5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:52.770150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-77a9fe93-7486-4e4b-8c03-659a138ad7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:52.777995 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 963/1928] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:52.782206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ec1b8-9601-4d83-87de-dcd4a847afbb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.791946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ec1b8-9601-4d83-87de-dcd4a847afbb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:52.796476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec04b1-34bf-45d4-a3df-3f72add48c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:52.797155 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec04b1-34bf-45d4-a3df-3f72add48c5d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.797436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec04b1-34bf-45d4-a3df-3f72add48c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.797709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec04b1-34bf-45d4-a3df-3f72add48c5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.798434 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec04b1-34bf-45d4-a3df-3f72add48c5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:52.798734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec04b1-34bf-45d4-a3df-3f72add48c5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:52.800352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec04b1-34bf-45d4-a3df-3f72add48c5d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:52.812455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 966/1929] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:52.816690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9fe93-7486-4e4b-8c03-659a138ad7f5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.822022 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:52.827349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9fe93-7486-4e4b-8c03-659a138ad7f5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:52.832212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb605d8-43fb-447f-a5ac-242b59b008f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:52.832908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb605d8-43fb-447f-a5ac-242b59b008f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:52.836278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb605d8-43fb-447f-a5ac-242b59b008f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:52.836278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb605d8-43fb-447f-a5ac-242b59b008f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:52.836278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb605d8-43fb-447f-a5ac-242b59b008f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:52.836278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb605d8-43fb-447f-a5ac-242b59b008f2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:52.969627 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 964/1930] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:52 2022] POST /identity/v3/projects => generated 393 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:52.971114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec04b1-34bf-45d4-a3df-3f72add48c5d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:52.982471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec04b1-34bf-45d4-a3df-3f72add48c5d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:53.038886 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:53.050472 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e0a16-b570-4fcd-a96c-2ccf58db6595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:53.051149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0a16-b570-4fcd-a96c-2ccf58db6595 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:53.051435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0a16-b570-4fcd-a96c-2ccf58db6595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:53.051713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0a16-b570-4fcd-a96c-2ccf58db6595 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:53.053043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e0a16-b570-4fcd-a96c-2ccf58db6595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:53.054941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e0a16-b570-4fcd-a96c-2ccf58db6595 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:53.153520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 967/1931] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:52 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:53.158375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb605d8-43fb-447f-a5ac-242b59b008f2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:53.169944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb605d8-43fb-447f-a5ac-242b59b008f2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:53.174755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406da4a-0586-4aba-80a1-6761c854ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:53.175601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406da4a-0586-4aba-80a1-6761c854ac38 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:53.176196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406da4a-0586-4aba-80a1-6761c854ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:53.176479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406da4a-0586-4aba-80a1-6761c854ac38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:53.177908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406da4a-0586-4aba-80a1-6761c854ac38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:53.179544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406da4a-0586-4aba-80a1-6761c854ac38 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:53.189806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 965/1932] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:53.194132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e0a16-b570-4fcd-a96c-2ccf58db6595 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:53.204850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e0a16-b570-4fcd-a96c-2ccf58db6595 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:53.210163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649b48d-650c-4de0-bbfd-838eb71ba736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:53.210839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649b48d-650c-4de0-bbfd-838eb71ba736 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:53.213396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649b48d-650c-4de0-bbfd-838eb71ba736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:53.213396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649b48d-650c-4de0-bbfd-838eb71ba736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:53.213396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649b48d-650c-4de0-bbfd-838eb71ba736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:53.214690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649b48d-650c-4de0-bbfd-838eb71ba736 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:53.226541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 968/1933] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:53.230921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406da4a-0586-4aba-80a1-6761c854ac38 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:53.242072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406da4a-0586-4aba-80a1-6761c854ac38 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:53.247921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1d26e-acbd-4253-8b05-6f01f6fc3fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:53.248933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1d26e-acbd-4253-8b05-6f01f6fc3fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:53.249327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1d26e-acbd-4253-8b05-6f01f6fc3fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:53.249671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1d26e-acbd-4253-8b05-6f01f6fc3fa5 admin admin] PATH_INFO: `/v3/projects/91ee9be2fdb44236911f0636b5b6b4e0/users/bf8704fab62845e0a4e40eecc2e52b76/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:53.268292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1d26e-acbd-4253-8b05-6f01f6fc3fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ee9be2fdb44236911f0636b5b6b4e0, user_id=bf8704fab62845e0a4e40eecc2e52b76, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:53.271034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1d26e-acbd-4253-8b05-6f01f6fc3fa5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:53.417240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 966/1934] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:53 2022] PUT /identity/v3/projects/91ee9be2fdb44236911f0636b5b6b4e0/users/bf8704fab62845e0a4e40eecc2e52b76/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:53.430619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf856ee-70b8-4db6-a352-88a9ab09725e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:53.430839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf856ee-70b8-4db6-a352-88a9ab09725e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:53.431174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf856ee-70b8-4db6-a352-88a9ab09725e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:53.459574 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-caf856ee-70b8-4db6-a352-88a9ab09725e None None] MFA Rules not processed for user `bf8704fab62845e0a4e40eecc2e52b76`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:53.460810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-caf856ee-70b8-4db6-a352-88a9ab09725e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:53.540105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 969/1935] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:53.550105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1d26e-acbd-4253-8b05-6f01f6fc3fa5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:53.623963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1d26e-acbd-4253-8b05-6f01f6fc3fa5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:53.628895 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563efbc-2f85-49bb-a9da-62b84a76cc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:53.629588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563efbc-2f85-49bb-a9da-62b84a76cc72 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:53.629887 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563efbc-2f85-49bb-a9da-62b84a76cc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:53.630174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563efbc-2f85-49bb-a9da-62b84a76cc72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:53.631000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563efbc-2f85-49bb-a9da-62b84a76cc72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:53.632998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563efbc-2f85-49bb-a9da-62b84a76cc72 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:53.731385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf856ee-70b8-4db6-a352-88a9ab09725e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:53.733083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-caf856ee-70b8-4db6-a352-88a9ab09725e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:53.748714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf856ee-70b8-4db6-a352-88a9ab09725e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:53.753957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f88e8f-2905-4630-9641-10772c46023c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:53.754899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f88e8f-2905-4630-9641-10772c46023c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:53.755341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f88e8f-2905-4630-9641-10772c46023c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:53.755767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f88e8f-2905-4630-9641-10772c46023c admin admin] PATH_INFO: `/v3/users/57f58ee106cf4dcb9014f6c118b9c7c4` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:53.757483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f88e8f-2905-4630-9641-10772c46023c admin admin] RBAC: Authorizing `identity:delete_user(user_id=57f58ee106cf4dcb9014f6c118b9c7c4)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:53.765358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f88e8f-2905-4630-9641-10772c46023c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:53.778517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 967/1936] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:53 2022] POST /identity/v3/projects => generated 393 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:50:53.784658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563efbc-2f85-49bb-a9da-62b84a76cc72 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:53.861153 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563efbc-2f85-49bb-a9da-62b84a76cc72 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:53.866585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce20df3-e016-4c45-9ede-0324c28b08e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:53.867261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce20df3-e016-4c45-9ede-0324c28b08e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:53.867542 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce20df3-e016-4c45-9ede-0324c28b08e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:53.867850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce20df3-e016-4c45-9ede-0324c28b08e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:53.868561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce20df3-e016-4c45-9ede-0324c28b08e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:53.870532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce20df3-e016-4c45-9ede-0324c28b08e1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:53.986076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 968/1937] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:53 2022] POST /identity/v3/users => generated 303 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:53.989560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce20df3-e016-4c45-9ede-0324c28b08e1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:54.002078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-70f88e8f-2905-4630-9641-10772c46023c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57f58ee106cf4dcb9014f6c118b9c7c4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:54.007838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce20df3-e016-4c45-9ede-0324c28b08e1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:54.017442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdc8a8-724a-4b0c-b49e-18550467c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:54.018170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdc8a8-724a-4b0c-b49e-18550467c84e admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:54.018870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdc8a8-724a-4b0c-b49e-18550467c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:54.019074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdc8a8-724a-4b0c-b49e-18550467c84e admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:54.019802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-70f88e8f-2905-4630-9641-10772c46023c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57f58ee106cf4dcb9014f6c118b9c7c4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:54.020109 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdc8a8-724a-4b0c-b49e-18550467c84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:54.022425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdc8a8-724a-4b0c-b49e-18550467c84e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:54.028556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-70f88e8f-2905-4630-9641-10772c46023c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57f58ee106cf4dcb9014f6c118b9c7c4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:54.036940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 969/1938] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:50:54.041103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdc8a8-724a-4b0c-b49e-18550467c84e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:54.053384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdc8a8-724a-4b0c-b49e-18550467c84e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:54.060010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb55182-0db0-4c7b-998f-487dcd2406c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:54.062778 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb55182-0db0-4c7b-998f-487dcd2406c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:54.062778 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb55182-0db0-4c7b-998f-487dcd2406c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:54.062778 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb55182-0db0-4c7b-998f-487dcd2406c4 admin admin] PATH_INFO: `/v3/projects/65800b8d6d494258be979728d4b9fdaf/users/fc25255b8ab549929d8b5ecb79253b83/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:54.065171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:54.078124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb55182-0db0-4c7b-998f-487dcd2406c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65800b8d6d494258be979728d4b9fdaf, user_id=fc25255b8ab549929d8b5ecb79253b83, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:54.079146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-70f88e8f-2905-4630-9641-10772c46023c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57f58ee106cf4dcb9014f6c118b9c7c4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:54.080077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-70f88e8f-2905-4630-9641-10772c46023c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57f58ee106cf4dcb9014f6c118b9c7c4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:54.080795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb55182-0db0-4c7b-998f-487dcd2406c4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:54.089005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 970/1939] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:53 2022] DELETE /identity/v3/users/57f58ee106cf4dcb9014f6c118b9c7c4 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:54.093554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b62c7-5dae-4bf5-8202-401dbb116145 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:54.093845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b62c7-5dae-4bf5-8202-401dbb116145 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:54.094206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b62c7-5dae-4bf5-8202-401dbb116145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:54.115972 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-494b62c7-5dae-4bf5-8202-401dbb116145 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:54.117390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-494b62c7-5dae-4bf5-8202-401dbb116145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:54.146420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:54.170684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 970/1940] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:54 2022] PUT /identity/v3/projects/65800b8d6d494258be979728d4b9fdaf/users/fc25255b8ab549929d8b5ecb79253b83/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:54.175623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb55182-0db0-4c7b-998f-487dcd2406c4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:54.177415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4eb55182-0db0-4c7b-998f-487dcd2406c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:54.187667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 971/1941] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:54.194879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b62c7-5dae-4bf5-8202-401dbb116145 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:54.196283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-494b62c7-5dae-4bf5-8202-401dbb116145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:54.262644 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb55182-0db0-4c7b-998f-487dcd2406c4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:54.264178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b62c7-5dae-4bf5-8202-401dbb116145 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:54.267487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7477b6-2482-4f38-a1c2-514d6a00f919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:54.268156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7477b6-2482-4f38-a1c2-514d6a00f919 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:54.268577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7477b6-2482-4f38-a1c2-514d6a00f919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:54.268856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7477b6-2482-4f38-a1c2-514d6a00f919 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:54.269645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7477b6-2482-4f38-a1c2-514d6a00f919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:54.271297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7477b6-2482-4f38-a1c2-514d6a00f919 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:54.302406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 971/1942] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:54 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:50:54.305857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7477b6-2482-4f38-a1c2-514d6a00f919 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:54.318552 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7477b6-2482-4f38-a1c2-514d6a00f919 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:54.320943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d878ee3-b57c-4fc2-a9ee-01edfdfb9fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:54.322360 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d878ee3-b57c-4fc2-a9ee-01edfdfb9fca service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:54.322697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d878ee3-b57c-4fc2-a9ee-01edfdfb9fca service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:54.322979 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d878ee3-b57c-4fc2-a9ee-01edfdfb9fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:54.324072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943ddaa-878f-442e-9d18-55541723b77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:54.325064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943ddaa-878f-442e-9d18-55541723b77f admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:54.325356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943ddaa-878f-442e-9d18-55541723b77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:54.325635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943ddaa-878f-442e-9d18-55541723b77f admin admin] PATH_INFO: `/v3/projects/65800b8d6d494258be979728d4b9fdaf/users/fc25255b8ab549929d8b5ecb79253b83/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:54.327635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943ddaa-878f-442e-9d18-55541723b77f admin admin] RBAC: Authorizing `identity:create_grant(project_id=65800b8d6d494258be979728d4b9fdaf, user_id=fc25255b8ab549929d8b5ecb79253b83, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:54.330358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943ddaa-878f-442e-9d18-55541723b77f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:54.340735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d878ee3-b57c-4fc2-a9ee-01edfdfb9fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:54.342406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d878ee3-b57c-4fc2-a9ee-01edfdfb9fca service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:54.348145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 972/1943] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:54 2022] PUT /identity/v3/projects/65800b8d6d494258be979728d4b9fdaf/users/fc25255b8ab549929d8b5ecb79253b83/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:54.353590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 972/1944] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:54.359499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9348e5-bc34-4ae2-92ca-8d48f5e099be None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:54.360000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9348e5-bc34-4ae2-92ca-8d48f5e099be None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:54.360348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9348e5-bc34-4ae2-92ca-8d48f5e099be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:54.397710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3d9348e5-bc34-4ae2-92ca-8d48f5e099be None None] MFA Rules not processed for user `fc25255b8ab549929d8b5ecb79253b83`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:54.398969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3d9348e5-bc34-4ae2-92ca-8d48f5e099be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:54.496039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 973/1945] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:54.510378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d878ee3-b57c-4fc2-a9ee-01edfdfb9fca service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:54.516773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4d878ee3-b57c-4fc2-a9ee-01edfdfb9fca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:54.547328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9348e5-bc34-4ae2-92ca-8d48f5e099be None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:54.548851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3d9348e5-bc34-4ae2-92ca-8d48f5e099be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:54.601675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d878ee3-b57c-4fc2-a9ee-01edfdfb9fca service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:54.663059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:54.687104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:54.691156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9348e5-bc34-4ae2-92ca-8d48f5e099be None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:54.696246 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:54.696418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355ebdb-ff03-4dda-8fca-caedeec4a819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:54.697001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ebdb-ff03-4dda-8fca-caedeec4a819 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:54.697171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:54.697458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ebdb-ff03-4dda-8fca-caedeec4a819 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:54.697571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:54.697744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355ebdb-ff03-4dda-8fca-caedeec4a819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:54.697845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] PATH_INFO: `/v3/users/67abe67c7af84c38998f10036c1bada4` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:54.699206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67abe67c7af84c38998f10036c1bada4)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:54.701294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:54.709218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355ebdb-ff03-4dda-8fca-caedeec4a819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:54.710789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355ebdb-ff03-4dda-8fca-caedeec4a819 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:54.728098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 973/1946] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:54.791092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355ebdb-ff03-4dda-8fca-caedeec4a819 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:54.793611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f355ebdb-ff03-4dda-8fca-caedeec4a819 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:54.807382 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355ebdb-ff03-4dda-8fca-caedeec4a819 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:54.814399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a375bf-50ce-44a9-89a7-8eb8cd9c451c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:54.815492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a375bf-50ce-44a9-89a7-8eb8cd9c451c service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:54.819638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a375bf-50ce-44a9-89a7-8eb8cd9c451c service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:54.819638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a375bf-50ce-44a9-89a7-8eb8cd9c451c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:54.819638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c4a375bf-50ce-44a9-89a7-8eb8cd9c451c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:54.904601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a375bf-50ce-44a9-89a7-8eb8cd9c451c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:54.905827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a375bf-50ce-44a9-89a7-8eb8cd9c451c service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:54.916572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 974/1947] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:54.926972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67abe67c7af84c38998f10036c1bada4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:54.940280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67abe67c7af84c38998f10036c1bada4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:54.957861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67abe67c7af84c38998f10036c1bada4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:55.021317 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67abe67c7af84c38998f10036c1bada4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:55.022204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67abe67c7af84c38998f10036c1bada4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:55.030812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 974/1948] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:54 2022] DELETE /identity/v3/users/67abe67c7af84c38998f10036c1bada4 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:55.035639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed240-150d-4124-8ef5-7bc23a6b2ffd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:55.036005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed240-150d-4124-8ef5-7bc23a6b2ffd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:55.037003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaed240-150d-4124-8ef5-7bc23a6b2ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:55.056957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-bfaed240-150d-4124-8ef5-7bc23a6b2ffd None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:55.058256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bfaed240-150d-4124-8ef5-7bc23a6b2ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:55.136824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 975/1949] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:55.155420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:55.159831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:55.276743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ec969-fae4-4b27-b3ee-de8d26c06d11 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:55.280575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b456f-b16e-4a6b-90dc-f92bd7fb7d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:55.281293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b456f-b16e-4a6b-90dc-f92bd7fb7d4f service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:55.281664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b456f-b16e-4a6b-90dc-f92bd7fb7d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:55.281931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b456f-b16e-4a6b-90dc-f92bd7fb7d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:55.293499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b456f-b16e-4a6b-90dc-f92bd7fb7d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:55.295551 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b456f-b16e-4a6b-90dc-f92bd7fb7d4f service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:55.307252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 975/1950] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:55.370644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaed240-150d-4124-8ef5-7bc23a6b2ffd None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:55.372403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bfaed240-150d-4124-8ef5-7bc23a6b2ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:55.388029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaed240-150d-4124-8ef5-7bc23a6b2ffd None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:55.400937 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:55.401844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:55.402186 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:55.402568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] PATH_INFO: `/v3/projects/61a80f5fe0794f679d4b5b0c7a4593ce` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:55.404786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a80f5fe0794f679d4b5b0c7a4593ce)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:55.410870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:55.610761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a80f5fe0794f679d4b5b0c7a4593ce', 'request_id': 'req-65fde33b-b46e-481a-a301-faf42195b84d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:55.838830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a80f5fe0794f679d4b5b0c7a4593ce', 'request_id': 'req-65fde33b-b46e-481a-a301-faf42195b84d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:55.857023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a80f5fe0794f679d4b5b0c7a4593ce', 'request_id': 'req-65fde33b-b46e-481a-a301-faf42195b84d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:55.901733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] The token cache is being invalidate because project 61a80f5fe0794f679d4b5b0c7a4593ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:55.902094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:55.904767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:55.906676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 976/1951] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:55 2022] DELETE /identity/v3/projects/61a80f5fe0794f679d4b5b0c7a4593ce => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:55.916387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b456f-b16e-4a6b-90dc-f92bd7fb7d4f service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:55.938518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b456f-b16e-4a6b-90dc-f92bd7fb7d4f service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:56.013910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:56.014732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:56.043682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:56.189163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fde33b-b46e-481a-a301-faf42195b84d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:56.205459 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:56.206344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:56.206625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:56.206897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] PATH_INFO: `/v3/projects/234f4c6ce6a84a37b1902871409807f3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:56.209006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=234f4c6ce6a84a37b1902871409807f3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:56.211049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:56.211756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0748e67-ccbc-4572-a5b1-bb40c527c526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:56.214800 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0748e67-ccbc-4572-a5b1-bb40c527c526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:56.215145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0748e67-ccbc-4572-a5b1-bb40c527c526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:56.215433 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0748e67-ccbc-4572-a5b1-bb40c527c526 admin admin] PATH_INFO: `/v3/users/6066f0c6dbad4921b84e0cc4d32e629a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:56.218527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0748e67-ccbc-4572-a5b1-bb40c527c526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6066f0c6dbad4921b84e0cc4d32e629a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:56.221123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0748e67-ccbc-4572-a5b1-bb40c527c526 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:56.257277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:56.483207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234f4c6ce6a84a37b1902871409807f3', 'request_id': 'req-c773e322-1134-4c2d-95d9-612df6094e3d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:56.486761 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e0748e67-ccbc-4572-a5b1-bb40c527c526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6066f0c6dbad4921b84e0cc4d32e629a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:56.501015 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e0748e67-ccbc-4572-a5b1-bb40c527c526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6066f0c6dbad4921b84e0cc4d32e629a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:56.528914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e0748e67-ccbc-4572-a5b1-bb40c527c526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6066f0c6dbad4921b84e0cc4d32e629a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:56.577328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234f4c6ce6a84a37b1902871409807f3', 'request_id': 'req-c773e322-1134-4c2d-95d9-612df6094e3d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:56.591528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234f4c6ce6a84a37b1902871409807f3', 'request_id': 'req-c773e322-1134-4c2d-95d9-612df6094e3d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:56.632190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] The token cache is being invalidate because project 234f4c6ce6a84a37b1902871409807f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:56.632744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:56.633539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c773e322-1134-4c2d-95d9-612df6094e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:56.635668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 977/1952] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:56 2022] DELETE /identity/v3/projects/234f4c6ce6a84a37b1902871409807f3 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:56.641227 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb858-3e32-48d7-8519-396cdf3a1a82 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:56.641553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb858-3e32-48d7-8519-396cdf3a1a82 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:56.641779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb858-3e32-48d7-8519-396cdf3a1a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:56.644822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e0748e67-ccbc-4572-a5b1-bb40c527c526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6066f0c6dbad4921b84e0cc4d32e629a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:56.645508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e0748e67-ccbc-4572-a5b1-bb40c527c526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6066f0c6dbad4921b84e0cc4d32e629a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:56.654272 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 976/1953] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:55 2022] DELETE /identity/v3/users/6066f0c6dbad4921b84e0cc4d32e629a => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:56.673695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ed8eb858-3e32-48d7-8519-396cdf3a1a82 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:56.676646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ed8eb858-3e32-48d7-8519-396cdf3a1a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:56.751319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 978/1954] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:56.759036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac820d6-482e-4f94-8f34-f113152dc1e2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:56.759539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac820d6-482e-4f94-8f34-f113152dc1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:56.759824 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac820d6-482e-4f94-8f34-f113152dc1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:56.778853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cac820d6-482e-4f94-8f34-f113152dc1e2 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:56.780240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cac820d6-482e-4f94-8f34-f113152dc1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:56.788974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 977/1955] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:56.793239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb858-3e32-48d7-8519-396cdf3a1a82 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:56.804128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb858-3e32-48d7-8519-396cdf3a1a82 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:56.809047 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb716d0-1aab-45ae-ac19-22e0ff048ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:56.809723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb716d0-1aab-45ae-ac19-22e0ff048ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:56.815377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb716d0-1aab-45ae-ac19-22e0ff048ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:56.815377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb716d0-1aab-45ae-ac19-22e0ff048ca4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:56.815377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb716d0-1aab-45ae-ac19-22e0ff048ca4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:56.815377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb716d0-1aab-45ae-ac19-22e0ff048ca4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:56.815377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb716d0-1aab-45ae-ac19-22e0ff048ca4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:56.827176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 979/1956] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:56.831636 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac820d6-482e-4f94-8f34-f113152dc1e2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:56.842630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac820d6-482e-4f94-8f34-f113152dc1e2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:56.847924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253682a4-1671-4673-9f71-a83d4dbfec23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:56.849170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253682a4-1671-4673-9f71-a83d4dbfec23 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:56.849463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253682a4-1671-4673-9f71-a83d4dbfec23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:56.849744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253682a4-1671-4673-9f71-a83d4dbfec23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:56.850466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253682a4-1671-4673-9f71-a83d4dbfec23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:56.852543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253682a4-1671-4673-9f71-a83d4dbfec23 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:56.938821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 978/1957] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:56 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:50:56.945469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb716d0-1aab-45ae-ac19-22e0ff048ca4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:56.956785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb716d0-1aab-45ae-ac19-22e0ff048ca4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:57.028062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb2189-4944-4ac9-8798-25f67bc198e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:57.028756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb2189-4944-4ac9-8798-25f67bc198e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.029075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb2189-4944-4ac9-8798-25f67bc198e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.029355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb2189-4944-4ac9-8798-25f67bc198e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.030058 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb2189-4944-4ac9-8798-25f67bc198e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:57.032183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb2189-4944-4ac9-8798-25f67bc198e7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:57.181393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 980/1958] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:56 2022] POST /identity/v3/users => generated 295 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:57.187487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253682a4-1671-4673-9f71-a83d4dbfec23 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:57.204725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:57.204850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253682a4-1671-4673-9f71-a83d4dbfec23 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:57.205689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb2189-4944-4ac9-8798-25f67bc198e7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:57.207287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ac9c3-390c-43d6-8556-289b52722547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:57.207584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-31eb2189-4944-4ac9-8798-25f67bc198e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:57.209058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ac9c3-390c-43d6-8556-289b52722547 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.209406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ac9c3-390c-43d6-8556-289b52722547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.209936 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ac9c3-390c-43d6-8556-289b52722547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.210865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ac9c3-390c-43d6-8556-289b52722547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:57.212850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ac9c3-390c-43d6-8556-289b52722547 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:57.222641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb2189-4944-4ac9-8798-25f67bc198e7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:57.231734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 979/1959] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:57.232836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ac9c3-390c-43d6-8556-289b52722547 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:57.240074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:57.240074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.240074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.240074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] PATH_INFO: `/v3/projects/5f1834b64f0f4341a9fb02c8534d54db` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.240512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f1834b64f0f4341a9fb02c8534d54db)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:57.240512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:57.245993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ac9c3-390c-43d6-8556-289b52722547 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:57.250041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db2bdc-9da3-4abc-831d-93425067f9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:57.250711 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db2bdc-9da3-4abc-831d-93425067f9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.250993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db2bdc-9da3-4abc-831d-93425067f9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.251279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db2bdc-9da3-4abc-831d-93425067f9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.252142 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db2bdc-9da3-4abc-831d-93425067f9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:57.253826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db2bdc-9da3-4abc-831d-93425067f9d3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:57.265050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 980/1960] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:57.269367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db2bdc-9da3-4abc-831d-93425067f9d3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:57.279585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db2bdc-9da3-4abc-831d-93425067f9d3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:57.284313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d90d6-dc56-4d2f-ab7a-d61b4a1698e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:57.285476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d90d6-dc56-4d2f-ab7a-d61b4a1698e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.285763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d90d6-dc56-4d2f-ab7a-d61b4a1698e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.286055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d90d6-dc56-4d2f-ab7a-d61b4a1698e1 admin admin] PATH_INFO: `/v3/projects/1a9f6af39f8d490c9a232f82d46d4d08/users/1d4ee9f78817409d9fc839ec3cc68f7b/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.303099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d90d6-dc56-4d2f-ab7a-d61b4a1698e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a9f6af39f8d490c9a232f82d46d4d08, user_id=1d4ee9f78817409d9fc839ec3cc68f7b, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:57.305944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d90d6-dc56-4d2f-ab7a-d61b4a1698e1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:57.413926 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 981/1961] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:57 2022] PUT /identity/v3/projects/1a9f6af39f8d490c9a232f82d46d4d08/users/1d4ee9f78817409d9fc839ec3cc68f7b/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:57.424541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff138ef-9ac9-4ff9-bf17-52936102f783 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.425097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff138ef-9ac9-4ff9-bf17-52936102f783 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.425692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff138ef-9ac9-4ff9-bf17-52936102f783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.431011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f1834b64f0f4341a9fb02c8534d54db', 'request_id': 'req-3cc5918d-e856-403c-a690-ce077cc12262'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:57.459730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0ff138ef-9ac9-4ff9-bf17-52936102f783 None None] MFA Rules not processed for user `1d4ee9f78817409d9fc839ec3cc68f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:57.461568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0ff138ef-9ac9-4ff9-bf17-52936102f783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:57.554039 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f1834b64f0f4341a9fb02c8534d54db', 'request_id': 'req-3cc5918d-e856-403c-a690-ce077cc12262'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:57.561090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 982/1962] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:57.561295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f1834b64f0f4341a9fb02c8534d54db', 'request_id': 'req-3cc5918d-e856-403c-a690-ce077cc12262'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:57.568725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff138ef-9ac9-4ff9-bf17-52936102f783 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:57.570470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0ff138ef-9ac9-4ff9-bf17-52936102f783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:57.575112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] The token cache is being invalidate because project 5f1834b64f0f4341a9fb02c8534d54db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:50:57.575406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:57.576119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3cc5918d-e856-403c-a690-ce077cc12262 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:50:57.578080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 981/1963] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:50:57 2022] DELETE /identity/v3/projects/5f1834b64f0f4341a9fb02c8534d54db => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:50:57.582601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:57.582792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf2d64-4915-4466-b558-642adffe7925 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.583094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf2d64-4915-4466-b558-642adffe7925 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.583380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf2d64-4915-4466-b558-642adffe7925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.610084 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a4bf2d64-4915-4466-b558-642adffe7925 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:57.611313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a4bf2d64-4915-4466-b558-642adffe7925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:57.651969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff138ef-9ac9-4ff9-bf17-52936102f783 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:57.656078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af2406-260f-484e-b622-a31a4027111f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:57.662385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af2406-260f-484e-b622-a31a4027111f service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.662385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af2406-260f-484e-b622-a31a4027111f service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.662385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af2406-260f-484e-b622-a31a4027111f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.662385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-79af2406-260f-484e-b622-a31a4027111f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:57.675001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af2406-260f-484e-b622-a31a4027111f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:57.678634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af2406-260f-484e-b622-a31a4027111f service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:57.692052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 983/1964] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:50:57.696901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 982/1965] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:57.706443 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3726-276a-4a15-99ef-6eb5d4955810 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.706825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3726-276a-4a15-99ef-6eb5d4955810 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.707168 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f3726-276a-4a15-99ef-6eb5d4955810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.728962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a94f3726-276a-4a15-99ef-6eb5d4955810 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:57.730395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a94f3726-276a-4a15-99ef-6eb5d4955810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:57.739438 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 984/1966] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:57.749815 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf2d64-4915-4466-b558-642adffe7925 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:57.761175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf2d64-4915-4466-b558-642adffe7925 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:57.766617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864787b3-644d-460f-8798-3a8e4156c7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:57.767374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864787b3-644d-460f-8798-3a8e4156c7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.770261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864787b3-644d-460f-8798-3a8e4156c7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.770261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864787b3-644d-460f-8798-3a8e4156c7c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.770261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864787b3-644d-460f-8798-3a8e4156c7c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:50:57.770261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864787b3-644d-460f-8798-3a8e4156c7c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:57.771310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864787b3-644d-460f-8798-3a8e4156c7c4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:57.786630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 983/1967] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:50:57.791722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f3726-276a-4a15-99ef-6eb5d4955810 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:57.804806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f3726-276a-4a15-99ef-6eb5d4955810 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:57.812579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac804ed-dccd-480c-a36b-2826fda192dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:57.813309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac804ed-dccd-480c-a36b-2826fda192dc admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.813725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac804ed-dccd-480c-a36b-2826fda192dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.814028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac804ed-dccd-480c-a36b-2826fda192dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.814858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac804ed-dccd-480c-a36b-2826fda192dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:57.816835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac804ed-dccd-480c-a36b-2826fda192dc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:57.906251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 985/1968] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:57 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:57.907359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864787b3-644d-460f-8798-3a8e4156c7c4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:57.918107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864787b3-644d-460f-8798-3a8e4156c7c4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:57.944776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:57.977527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711fe19-3f7e-41f6-b189-9a724e661df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:57.978219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711fe19-3f7e-41f6-b189-9a724e661df3 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:57.978511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711fe19-3f7e-41f6-b189-9a724e661df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:57.978795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711fe19-3f7e-41f6-b189-9a724e661df3 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:57.979509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711fe19-3f7e-41f6-b189-9a724e661df3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:57.981488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711fe19-3f7e-41f6-b189-9a724e661df3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:58.079369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 984/1969] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:57 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:50:58.083575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac804ed-dccd-480c-a36b-2826fda192dc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:58.094422 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac804ed-dccd-480c-a36b-2826fda192dc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:58.099724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735588b3-0378-426e-bd19-573335b02575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:58.100357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735588b3-0378-426e-bd19-573335b02575 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:58.100639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735588b3-0378-426e-bd19-573335b02575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:58.100946 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735588b3-0378-426e-bd19-573335b02575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:58.101692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735588b3-0378-426e-bd19-573335b02575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:58.103299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735588b3-0378-426e-bd19-573335b02575 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:58.113463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 986/1970] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:58.118313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711fe19-3f7e-41f6-b189-9a724e661df3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:58.128226 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711fe19-3f7e-41f6-b189-9a724e661df3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:58.133115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1114b-85b5-4149-a572-0cd60acfa834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:58.133788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1114b-85b5-4149-a572-0cd60acfa834 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:58.134076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1114b-85b5-4149-a572-0cd60acfa834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:58.134414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1114b-85b5-4149-a572-0cd60acfa834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:58.135054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1114b-85b5-4149-a572-0cd60acfa834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:58.136663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1114b-85b5-4149-a572-0cd60acfa834 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:58.149811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 985/1971] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:58.157375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735588b3-0378-426e-bd19-573335b02575 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:58.168408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735588b3-0378-426e-bd19-573335b02575 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:58.173210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cb12b-cb3e-4d2c-9354-8ea26ce91af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:58.174131 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cb12b-cb3e-4d2c-9354-8ea26ce91af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:58.174404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cb12b-cb3e-4d2c-9354-8ea26ce91af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:58.174673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cb12b-cb3e-4d2c-9354-8ea26ce91af8 admin admin] PATH_INFO: `/v3/projects/cd57871edf8b400ba5c86b87ff09732f/users/d7ff1683087e4b7192039a837d5840f1/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:58.190818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cb12b-cb3e-4d2c-9354-8ea26ce91af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd57871edf8b400ba5c86b87ff09732f, user_id=d7ff1683087e4b7192039a837d5840f1, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:58.193606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cb12b-cb3e-4d2c-9354-8ea26ce91af8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:58.280470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 987/1972] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:58 2022] PUT /identity/v3/projects/cd57871edf8b400ba5c86b87ff09732f/users/d7ff1683087e4b7192039a837d5840f1/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:58.292661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357dfbe-9cb1-4c01-a420-99652f880b0c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:58.292980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357dfbe-9cb1-4c01-a420-99652f880b0c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:58.293262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357dfbe-9cb1-4c01-a420-99652f880b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:58.321311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f357dfbe-9cb1-4c01-a420-99652f880b0c None None] MFA Rules not processed for user `d7ff1683087e4b7192039a837d5840f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:58.322554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f357dfbe-9cb1-4c01-a420-99652f880b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:58.391127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 986/1973] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:50:58.396067 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cb12b-cb3e-4d2c-9354-8ea26ce91af8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:58.466058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cb12b-cb3e-4d2c-9354-8ea26ce91af8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:58.471161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11641206-6bd4-495b-9388-dff739f42ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:58.472129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641206-6bd4-495b-9388-dff739f42ece admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:58.472593 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641206-6bd4-495b-9388-dff739f42ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:58.472904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641206-6bd4-495b-9388-dff739f42ece admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:58.474298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11641206-6bd4-495b-9388-dff739f42ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:58.476329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11641206-6bd4-495b-9388-dff739f42ece admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:58.577826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 988/1974] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:50:58 2022] POST /identity/v3/projects => generated 381 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:58.582007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357dfbe-9cb1-4c01-a420-99652f880b0c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:58.594841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357dfbe-9cb1-4c01-a420-99652f880b0c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:58.652242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2c02a-dd8d-4f8c-af15-c9f433422ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:58.652987 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2c02a-dd8d-4f8c-af15-c9f433422ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:58.653271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2c02a-dd8d-4f8c-af15-c9f433422ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:58.653544 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2c02a-dd8d-4f8c-af15-c9f433422ca1 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:58.654228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2c02a-dd8d-4f8c-af15-c9f433422ca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:58.656241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2c02a-dd8d-4f8c-af15-c9f433422ca1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:58.737558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 987/1975] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:50:58 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:50:58.741241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11641206-6bd4-495b-9388-dff739f42ece admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:58.754390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11641206-6bd4-495b-9388-dff739f42ece admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:58.764571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43b03e-5284-4026-8b3b-503c0cd716d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:58.765827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43b03e-5284-4026-8b3b-503c0cd716d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:58.766180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43b03e-5284-4026-8b3b-503c0cd716d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:58.766460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43b03e-5284-4026-8b3b-503c0cd716d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:58.767892 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43b03e-5284-4026-8b3b-503c0cd716d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:58.769460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43b03e-5284-4026-8b3b-503c0cd716d3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:58.782569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 989/1976] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:58.787285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2c02a-dd8d-4f8c-af15-c9f433422ca1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:58.799429 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2c02a-dd8d-4f8c-af15-c9f433422ca1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:58.804765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0105db-524b-4553-84ea-1507d77efff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:58.805967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0105db-524b-4553-84ea-1507d77efff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:58.806278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0105db-524b-4553-84ea-1507d77efff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:58.806630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0105db-524b-4553-84ea-1507d77efff7 admin admin] PATH_INFO: `/v3/projects/69b79ddf69f741d7bb91ed2770cee020/users/9b6e741d81be4bc8af6d51a309998847/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:58.827669 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0105db-524b-4553-84ea-1507d77efff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b79ddf69f741d7bb91ed2770cee020, user_id=9b6e741d81be4bc8af6d51a309998847, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:58.830738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0105db-524b-4553-84ea-1507d77efff7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:58.904296 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:58.911574 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 988/1977] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:58 2022] PUT /identity/v3/projects/69b79ddf69f741d7bb91ed2770cee020/users/9b6e741d81be4bc8af6d51a309998847/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:58.915549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43b03e-5284-4026-8b3b-503c0cd716d3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:58.925824 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43b03e-5284-4026-8b3b-503c0cd716d3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:58.946101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:50:58.984187 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fa99e-e90a-44aa-9369-615c82fa4646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:58.984853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fa99e-e90a-44aa-9369-615c82fa4646 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:58.985164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fa99e-e90a-44aa-9369-615c82fa4646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:58.985442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fa99e-e90a-44aa-9369-615c82fa4646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:58.986167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fa99e-e90a-44aa-9369-615c82fa4646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:58.987809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fa99e-e90a-44aa-9369-615c82fa4646 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:58.997622 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 990/1978] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:50:59.001873 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0105db-524b-4553-84ea-1507d77efff7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:59.013319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0105db-524b-4553-84ea-1507d77efff7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:59.018014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb853b73-09fe-4c2e-94e4-e0ec732c0462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:59.018902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb853b73-09fe-4c2e-94e4-e0ec732c0462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:59.019188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb853b73-09fe-4c2e-94e4-e0ec732c0462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:59.019465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb853b73-09fe-4c2e-94e4-e0ec732c0462 admin admin] PATH_INFO: `/v3/projects/69b79ddf69f741d7bb91ed2770cee020/users/9b6e741d81be4bc8af6d51a309998847/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:59.022120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb853b73-09fe-4c2e-94e4-e0ec732c0462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69b79ddf69f741d7bb91ed2770cee020, user_id=9b6e741d81be4bc8af6d51a309998847, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:59.024814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb853b73-09fe-4c2e-94e4-e0ec732c0462 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:59.035014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 989/1979] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:50:59 2022] PUT /identity/v3/projects/69b79ddf69f741d7bb91ed2770cee020/users/9b6e741d81be4bc8af6d51a309998847/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:50:59.046484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee26786-2d31-4d88-9e52-17f5f6080e7e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:59.046832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee26786-2d31-4d88-9e52-17f5f6080e7e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:59.047113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee26786-2d31-4d88-9e52-17f5f6080e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:59.073354 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bee26786-2d31-4d88-9e52-17f5f6080e7e None None] MFA Rules not processed for user `9b6e741d81be4bc8af6d51a309998847`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:50:59.074555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bee26786-2d31-4d88-9e52-17f5f6080e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:59.158368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 991/1980] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:50:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:50:59.178330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb853b73-09fe-4c2e-94e4-e0ec732c0462 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:50:59.180444 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eb853b73-09fe-4c2e-94e4-e0ec732c0462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:50:59.288023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb853b73-09fe-4c2e-94e4-e0ec732c0462 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:50:59.295418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5e6ce-7378-4408-96df-995e63547025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:50:59.296225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e6ce-7378-4408-96df-995e63547025 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:50:59.296743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e6ce-7378-4408-96df-995e63547025 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:50:59.297059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5e6ce-7378-4408-96df-995e63547025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:50:59.312680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5e6ce-7378-4408-96df-995e63547025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:50:59.315029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b5e6ce-7378-4408-96df-995e63547025 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:50:59.332685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 990/1981] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:50:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:50:59.974323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee26786-2d31-4d88-9e52-17f5f6080e7e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:00.068730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee26786-2d31-4d88-9e52-17f5f6080e7e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:00.075431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:00.076367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:00.076918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:00.077098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] PATH_INFO: `/v3/users/d7ff1683087e4b7192039a837d5840f1` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:00.078486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7ff1683087e4b7192039a837d5840f1)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:00.080806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:00.265766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:00.268616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7ff1683087e4b7192039a837d5840f1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:00.276243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7ff1683087e4b7192039a837d5840f1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:00.288480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7ff1683087e4b7192039a837d5840f1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:00.375914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7ff1683087e4b7192039a837d5840f1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:00.376574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7ff1683087e4b7192039a837d5840f1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:00.386372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 992/1982] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:50:59 2022] DELETE /identity/v3/users/d7ff1683087e4b7192039a837d5840f1 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:00.393354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552c3fa-6352-4b8a-bacb-b36437b434dc None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:00.393896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552c3fa-6352-4b8a-bacb-b36437b434dc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:00.394261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552c3fa-6352-4b8a-bacb-b36437b434dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:00.425637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c552c3fa-6352-4b8a-bacb-b36437b434dc None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:00.426968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c552c3fa-6352-4b8a-bacb-b36437b434dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:00.499479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 991/1983] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:00.512712 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:00.514229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:00.595673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6561d-207c-4e36-b2d0-781bc41a1261 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:00.601044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fea4f6-8ef9-4e59-9a44-a8c146fcbe23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:00.617760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fea4f6-8ef9-4e59-9a44-a8c146fcbe23 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:00.617760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fea4f6-8ef9-4e59-9a44-a8c146fcbe23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:00.617760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fea4f6-8ef9-4e59-9a44-a8c146fcbe23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:00.618266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fea4f6-8ef9-4e59-9a44-a8c146fcbe23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:00.620459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fea4f6-8ef9-4e59-9a44-a8c146fcbe23 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:00.642392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 993/1984] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:01.155039 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552c3fa-6352-4b8a-bacb-b36437b434dc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:01.156565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c552c3fa-6352-4b8a-bacb-b36437b434dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:01.185020 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552c3fa-6352-4b8a-bacb-b36437b434dc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:01.191388 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:01.192780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:01.193105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:01.193389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] PATH_INFO: `/v3/projects/cd57871edf8b400ba5c86b87ff09732f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:01.195751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd57871edf8b400ba5c86b87ff09732f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:01.197760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:01.368218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd57871edf8b400ba5c86b87ff09732f', 'request_id': 'req-451ed432-6446-4d52-85d1-da3c151d516c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.472941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd57871edf8b400ba5c86b87ff09732f', 'request_id': 'req-451ed432-6446-4d52-85d1-da3c151d516c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.483505 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd57871edf8b400ba5c86b87ff09732f', 'request_id': 'req-451ed432-6446-4d52-85d1-da3c151d516c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.501255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] The token cache is being invalidate because project cd57871edf8b400ba5c86b87ff09732f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:01.501556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.502238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.503848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 992/1985] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:01 2022] DELETE /identity/v3/projects/cd57871edf8b400ba5c86b87ff09732f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:01.519629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fea4f6-8ef9-4e59-9a44-a8c146fcbe23 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:01.529670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fea4f6-8ef9-4e59-9a44-a8c146fcbe23 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:01.596737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cbe005-1a27-4b32-9e23-e63d591ce033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:01.597672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbe005-1a27-4b32-9e23-e63d591ce033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:01.597957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbe005-1a27-4b32-9e23-e63d591ce033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:01.598239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cbe005-1a27-4b32-9e23-e63d591ce033 admin admin] PATH_INFO: `/v3/users/9b6e741d81be4bc8af6d51a309998847` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:01.599866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:01.601309 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:01.610810 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cbe005-1a27-4b32-9e23-e63d591ce033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b6e741d81be4bc8af6d51a309998847)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:01.614704 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cbe005-1a27-4b32-9e23-e63d591ce033 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:01.620730 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ed432-6446-4d52-85d1-da3c151d516c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:01.630830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2d219-ae80-4dfd-a3fe-35a7b94eaa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:01.632262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2d219-ae80-4dfd-a3fe-35a7b94eaa2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:01.633331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2d219-ae80-4dfd-a3fe-35a7b94eaa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:01.633798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2d219-ae80-4dfd-a3fe-35a7b94eaa2d admin admin] PATH_INFO: `/v3/users/ed634dbd0a514cb08019d9dc08c79362` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:01.637205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2d219-ae80-4dfd-a3fe-35a7b94eaa2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed634dbd0a514cb08019d9dc08c79362)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:01.645018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2d219-ae80-4dfd-a3fe-35a7b94eaa2d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:01.653160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:01.862383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7ef2d219-ae80-4dfd-a3fe-35a7b94eaa2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed634dbd0a514cb08019d9dc08c79362'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.879337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-20cbe005-1a27-4b32-9e23-e63d591ce033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b6e741d81be4bc8af6d51a309998847'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.893198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7ef2d219-ae80-4dfd-a3fe-35a7b94eaa2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed634dbd0a514cb08019d9dc08c79362'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.902095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7ef2d219-ae80-4dfd-a3fe-35a7b94eaa2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed634dbd0a514cb08019d9dc08c79362'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.905834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-20cbe005-1a27-4b32-9e23-e63d591ce033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b6e741d81be4bc8af6d51a309998847'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.923581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:01.947989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-20cbe005-1a27-4b32-9e23-e63d591ce033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b6e741d81be4bc8af6d51a309998847'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.975916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7ef2d219-ae80-4dfd-a3fe-35a7b94eaa2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed634dbd0a514cb08019d9dc08c79362'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.977393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7ef2d219-ae80-4dfd-a3fe-35a7b94eaa2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed634dbd0a514cb08019d9dc08c79362'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:01.987847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 993/1986] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:01 2022] DELETE /identity/v3/users/ed634dbd0a514cb08019d9dc08c79362 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:01.990016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291117e-753b-42e9-8f88-ca4975cd694b None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:01.990389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291117e-753b-42e9-8f88-ca4975cd694b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:01.990740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291117e-753b-42e9-8f88-ca4975cd694b None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:01.992557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 994/1987] 10.222.0.88 () {54 vars in 921 bytes} [Mon Jul 18 16:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:51:01.993611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71611c2f-d863-4f26-abcb-289cb4386ba7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:01.993968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71611c2f-d863-4f26-abcb-289cb4386ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:01.994318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71611c2f-d863-4f26-abcb-289cb4386ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.013275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-71611c2f-d863-4f26-abcb-289cb4386ba7 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:02.014512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-71611c2f-d863-4f26-abcb-289cb4386ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:02.038713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-20cbe005-1a27-4b32-9e23-e63d591ce033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b6e741d81be4bc8af6d51a309998847'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.039570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-20cbe005-1a27-4b32-9e23-e63d591ce033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b6e741d81be4bc8af6d51a309998847'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.048381 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 994/1988] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:01 2022] DELETE /identity/v3/users/9b6e741d81be4bc8af6d51a309998847 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:02.050532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3c9e-973e-419c-b81a-7da47b862dd7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.052487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3c9e-973e-419c-b81a-7da47b862dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.053025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3c9e-973e-419c-b81a-7da47b862dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.091740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6dfe3c9e-973e-419c-b81a-7da47b862dd7 None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:02.093297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6dfe3c9e-973e-419c-b81a-7da47b862dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:02.120927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 995/1989] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:02.136729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71611c2f-d863-4f26-abcb-289cb4386ba7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:02.138552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-71611c2f-d863-4f26-abcb-289cb4386ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:02.210653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 995/1990] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Jul 18 16:51:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:02.226826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe3c9e-973e-419c-b81a-7da47b862dd7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:02.228240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6dfe3c9e-973e-419c-b81a-7da47b862dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:02.232036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71611c2f-d863-4f26-abcb-289cb4386ba7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:02.244078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe3c9e-973e-419c-b81a-7da47b862dd7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:02.250134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12942d-ee9b-4369-b216-305f289eb7ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:02.250828 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12942d-ee9b-4369-b216-305f289eb7ad service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.251093 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12942d-ee9b-4369-b216-305f289eb7ad service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.251378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12942d-ee9b-4369-b216-305f289eb7ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.262393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12942d-ee9b-4369-b216-305f289eb7ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:02.264025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12942d-ee9b-4369-b216-305f289eb7ad service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:02.276403 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 996/1991] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:02.318877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b17e3d-db75-434d-bfd7-621b657d80cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:02.319484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b17e3d-db75-434d-bfd7-621b657d80cf service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.319842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b17e3d-db75-434d-bfd7-621b657d80cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.320219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b17e3d-db75-434d-bfd7-621b657d80cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.406887 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b17e3d-db75-434d-bfd7-621b657d80cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:02.409761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b17e3d-db75-434d-bfd7-621b657d80cf service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:02.424736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 996/1992] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:02.428243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:02.777601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12942d-ee9b-4369-b216-305f289eb7ad service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:02.779280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ae12942d-ee9b-4369-b216-305f289eb7ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:02.796567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12942d-ee9b-4369-b216-305f289eb7ad service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:02.801717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:02.802660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.802955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.810149 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] PATH_INFO: `/v3/projects/69b79ddf69f741d7bb91ed2770cee020` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.810149 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69b79ddf69f741d7bb91ed2770cee020)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:02.810149 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:02.910053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69b79ddf69f741d7bb91ed2770cee020', 'request_id': 'req-9e7ffca2-e97c-4a4f-b470-68db1dad7659'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.916708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69b79ddf69f741d7bb91ed2770cee020', 'request_id': 'req-9e7ffca2-e97c-4a4f-b470-68db1dad7659'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.922943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69b79ddf69f741d7bb91ed2770cee020', 'request_id': 'req-9e7ffca2-e97c-4a4f-b470-68db1dad7659'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.969899 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] The token cache is being invalidate because project 69b79ddf69f741d7bb91ed2770cee020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:02.970196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.970844 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:02.972464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 997/1993] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:02 2022] DELETE /identity/v3/projects/69b79ddf69f741d7bb91ed2770cee020 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:02.977351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce602e46-d2db-416e-a7ec-7ed2bc5034fd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:02.977796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce602e46-d2db-416e-a7ec-7ed2bc5034fd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:02.977908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce602e46-d2db-416e-a7ec-7ed2bc5034fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:02.998229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ce602e46-d2db-416e-a7ec-7ed2bc5034fd None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:02.999446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ce602e46-d2db-416e-a7ec-7ed2bc5034fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:03.029555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:03.031357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:03.083846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 997/1994] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:03.091785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6898d-a799-48a3-8393-efdb34ec0e5c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.092081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6898d-a799-48a3-8393-efdb34ec0e5c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.092392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6898d-a799-48a3-8393-efdb34ec0e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.093795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:03.110672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ffca2-e97c-4a4f-b470-68db1dad7659 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:03.116112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:03.117009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.117153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-01d6898d-a799-48a3-8393-efdb34ec0e5c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:03.117465 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.118131 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] PATH_INFO: `/v3/projects/81bc6971b06b440182be7c4f31b9d565` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.118457 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-01d6898d-a799-48a3-8393-efdb34ec0e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:03.118965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81bc6971b06b440182be7c4f31b9d565)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:03.120916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:03.127551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 998/1995] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:03.134016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6898d-a799-48a3-8393-efdb34ec0e5c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:03.147026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6898d-a799-48a3-8393-efdb34ec0e5c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:03.152310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac3a82-77e2-4c7a-95f5-d94d973b323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:03.153046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac3a82-77e2-4c7a-95f5-d94d973b323d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.153334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac3a82-77e2-4c7a-95f5-d94d973b323d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.153615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac3a82-77e2-4c7a-95f5-d94d973b323d admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.154364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac3a82-77e2-4c7a-95f5-d94d973b323d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:03.154677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac3a82-77e2-4c7a-95f5-d94d973b323d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:03.156290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac3a82-77e2-4c7a-95f5-d94d973b323d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:03.169384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 999/1996] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:03.174294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac3a82-77e2-4c7a-95f5-d94d973b323d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:03.184887 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac3a82-77e2-4c7a-95f5-d94d973b323d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:03.189678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93403601-a661-4047-81cf-92b1168c70f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:03.190330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93403601-a661-4047-81cf-92b1168c70f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.190617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93403601-a661-4047-81cf-92b1168c70f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.190897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93403601-a661-4047-81cf-92b1168c70f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.191585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93403601-a661-4047-81cf-92b1168c70f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:03.194125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93403601-a661-4047-81cf-92b1168c70f2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:03.284721 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81bc6971b06b440182be7c4f31b9d565', 'request_id': 'req-efae08a9-2b19-4ee7-a87b-46827c7c24b1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:03.291056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81bc6971b06b440182be7c4f31b9d565', 'request_id': 'req-efae08a9-2b19-4ee7-a87b-46827c7c24b1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:03.296708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81bc6971b06b440182be7c4f31b9d565', 'request_id': 'req-efae08a9-2b19-4ee7-a87b-46827c7c24b1'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:03.338367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1000/1997] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:03 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:03.341365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93403601-a661-4047-81cf-92b1168c70f2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:03.392468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:03.403449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93403601-a661-4047-81cf-92b1168c70f2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:03.408938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] The token cache is being invalidate because project 81bc6971b06b440182be7c4f31b9d565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:03.409235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:03.412934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:03.413042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb722954-1fc7-4892-a1ca-5f85754629cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:03.413149 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 998/1998] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:03 2022] DELETE /identity/v3/projects/81bc6971b06b440182be7c4f31b9d565 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:03.413254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb722954-1fc7-4892-a1ca-5f85754629cb admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.413254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb722954-1fc7-4892-a1ca-5f85754629cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.413254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb722954-1fc7-4892-a1ca-5f85754629cb admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.413602 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb722954-1fc7-4892-a1ca-5f85754629cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:03.415593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb722954-1fc7-4892-a1ca-5f85754629cb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:03.416145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:03.417619 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:03.482396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae08a9-2b19-4ee7-a87b-46827c7c24b1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:03.486973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:03.487846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.488172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.488478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] PATH_INFO: `/v3/users/10d2c3c2ba1243978cbccec6174a51e4` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.490343 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] RBAC: Authorizing `identity:delete_user(user_id=10d2c3c2ba1243978cbccec6174a51e4)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:03.492262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:03.507645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1001/1999] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:03 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:03.513078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb722954-1fc7-4892-a1ca-5f85754629cb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:03.514744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cb722954-1fc7-4892-a1ca-5f85754629cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:03.527424 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb722954-1fc7-4892-a1ca-5f85754629cb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:03.532391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cddae-9571-42bb-9a74-ae1e57f5639a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:03.533095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cddae-9571-42bb-9a74-ae1e57f5639a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.533381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cddae-9571-42bb-9a74-ae1e57f5639a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.533857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cddae-9571-42bb-9a74-ae1e57f5639a admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.534508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cddae-9571-42bb-9a74-ae1e57f5639a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:03.536697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cddae-9571-42bb-9a74-ae1e57f5639a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:03.546782 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1002/2000] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:03.550729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cddae-9571-42bb-9a74-ae1e57f5639a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:03.561240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cddae-9571-42bb-9a74-ae1e57f5639a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:03.566877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea17e06-b111-43cf-a69b-96b5d9ba7155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:03.567546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea17e06-b111-43cf-a69b-96b5d9ba7155 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.567842 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea17e06-b111-43cf-a69b-96b5d9ba7155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.568126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea17e06-b111-43cf-a69b-96b5d9ba7155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.568858 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea17e06-b111-43cf-a69b-96b5d9ba7155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:03.570506 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea17e06-b111-43cf-a69b-96b5d9ba7155 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:03.581347 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1003/2001] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:03.585233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea17e06-b111-43cf-a69b-96b5d9ba7155 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:03.595471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea17e06-b111-43cf-a69b-96b5d9ba7155 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:03.605243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a005e96-aa05-4d2f-845c-319ecded5fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:03.605243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005e96-aa05-4d2f-845c-319ecded5fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.605243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005e96-aa05-4d2f-845c-319ecded5fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.605243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005e96-aa05-4d2f-845c-319ecded5fe3 admin admin] PATH_INFO: `/v3/projects/0b8636400dd840e7b5b6dceb7c31d5f5/users/4ab8aeaeb8364d359b9e28386e78e19e/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.618636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a005e96-aa05-4d2f-845c-319ecded5fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8636400dd840e7b5b6dceb7c31d5f5, user_id=4ab8aeaeb8364d359b9e28386e78e19e, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:03.621396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a005e96-aa05-4d2f-845c-319ecded5fe3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:03.669342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10d2c3c2ba1243978cbccec6174a51e4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:03.677475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10d2c3c2ba1243978cbccec6174a51e4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:03.691514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10d2c3c2ba1243978cbccec6174a51e4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:03.741313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1004/2002] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:03 2022] PUT /identity/v3/projects/0b8636400dd840e7b5b6dceb7c31d5f5/users/4ab8aeaeb8364d359b9e28386e78e19e/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:03.752909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10d2c3c2ba1243978cbccec6174a51e4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:03.753933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10d2c3c2ba1243978cbccec6174a51e4'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:03.754526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9168d42-16fe-499f-93b4-6b775fe9cf8e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.754825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9168d42-16fe-499f-93b4-6b775fe9cf8e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.755111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9168d42-16fe-499f-93b4-6b775fe9cf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.778196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 999/2003] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:03 2022] DELETE /identity/v3/users/10d2c3c2ba1243978cbccec6174a51e4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:03.791392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f9168d42-16fe-499f-93b4-6b775fe9cf8e None None] MFA Rules not processed for user `4ab8aeaeb8364d359b9e28386e78e19e`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:03.792700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f9168d42-16fe-499f-93b4-6b775fe9cf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:03.888980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1005/2004] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:03.893876 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:03.896065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:03.978704 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c6ec97-aec7-4dc3-ab7a-25996482023b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:03.984771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938285d-6419-4eec-8c49-98f37676dd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:03.985515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938285d-6419-4eec-8c49-98f37676dd1e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:03.985886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938285d-6419-4eec-8c49-98f37676dd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:03.986163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938285d-6419-4eec-8c49-98f37676dd1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:03.987248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938285d-6419-4eec-8c49-98f37676dd1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:03.988934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4938285d-6419-4eec-8c49-98f37676dd1e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:04.008024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:04.097874 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1000/2005] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:03 2022] POST /identity/v3/projects => generated 383 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:04.103184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9168d42-16fe-499f-93b4-6b775fe9cf8e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:04.114998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9168d42-16fe-499f-93b4-6b775fe9cf8e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:04.179778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4e561-a394-4f5a-9027-5090e4dc55af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:04.180465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4e561-a394-4f5a-9027-5090e4dc55af admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:04.180753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4e561-a394-4f5a-9027-5090e4dc55af admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:04.181065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4e561-a394-4f5a-9027-5090e4dc55af admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:04.188283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4e561-a394-4f5a-9027-5090e4dc55af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:04.188283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4e561-a394-4f5a-9027-5090e4dc55af admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:04.206424 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938285d-6419-4eec-8c49-98f37676dd1e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:04.209106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4938285d-6419-4eec-8c49-98f37676dd1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:04.223777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938285d-6419-4eec-8c49-98f37676dd1e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:04.228977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:04.229903 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:04.230217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:04.230497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] PATH_INFO: `/v3/projects/a1207a3917fa48e98b5afada1afda468` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:04.231743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1207a3917fa48e98b5afada1afda468)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:04.234374 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:04.292563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1006/2006] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:04 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:04.297756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4e561-a394-4f5a-9027-5090e4dc55af admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:04.311024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4e561-a394-4f5a-9027-5090e4dc55af admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:04.326189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc566ce8-1f3e-4baf-b3dc-90f0419935e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:04.327087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc566ce8-1f3e-4baf-b3dc-90f0419935e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:04.327294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc566ce8-1f3e-4baf-b3dc-90f0419935e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:04.332361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc566ce8-1f3e-4baf-b3dc-90f0419935e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:04.332361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc566ce8-1f3e-4baf-b3dc-90f0419935e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:04.332361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc566ce8-1f3e-4baf-b3dc-90f0419935e6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:04.343590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1007/2007] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:04.347713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc566ce8-1f3e-4baf-b3dc-90f0419935e6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:04.354212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:04.362029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc566ce8-1f3e-4baf-b3dc-90f0419935e6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:04.388631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1207a3917fa48e98b5afada1afda468', 'request_id': 'req-3d880704-88d9-4f01-b096-a445a88faaf6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:04.398192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1207a3917fa48e98b5afada1afda468', 'request_id': 'req-3d880704-88d9-4f01-b096-a445a88faaf6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:04.404749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1207a3917fa48e98b5afada1afda468', 'request_id': 'req-3d880704-88d9-4f01-b096-a445a88faaf6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:04.453909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab986b86-88eb-4459-be93-39fde5aed5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:04.455508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab986b86-88eb-4459-be93-39fde5aed5e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:04.456043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab986b86-88eb-4459-be93-39fde5aed5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:04.458227 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab986b86-88eb-4459-be93-39fde5aed5e6 admin admin] PATH_INFO: `/v3/projects/d493936e436640079d33418a420d671c/users/060e35e1df4940adb6b569b16fef779f/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:04.483059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab986b86-88eb-4459-be93-39fde5aed5e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d493936e436640079d33418a420d671c, user_id=060e35e1df4940adb6b569b16fef779f, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:04.485998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab986b86-88eb-4459-be93-39fde5aed5e6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:04.500124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] The token cache is being invalidate because project a1207a3917fa48e98b5afada1afda468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:04.500484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:04.501886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3d880704-88d9-4f01-b096-a445a88faaf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:04.504298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1001/2008] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:04 2022] DELETE /identity/v3/projects/a1207a3917fa48e98b5afada1afda468 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:04.508872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ed1ad-40c7-4053-a816-53f32f2a01aa None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:04.509185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ed1ad-40c7-4053-a816-53f32f2a01aa None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:04.509470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ed1ad-40c7-4053-a816-53f32f2a01aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:04.530245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d46ed1ad-40c7-4053-a816-53f32f2a01aa None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:04.532283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d46ed1ad-40c7-4053-a816-53f32f2a01aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:04.542385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1002/2009] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:04.550155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaae6e2-b156-4df2-8c3b-b36a9ccf7d8a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:04.550449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaae6e2-b156-4df2-8c3b-b36a9ccf7d8a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:04.550734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaae6e2-b156-4df2-8c3b-b36a9ccf7d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:04.569546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-fcaae6e2-b156-4df2-8c3b-b36a9ccf7d8a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:04.570687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fcaae6e2-b156-4df2-8c3b-b36a9ccf7d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:04.578980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1003/2010] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:04.583811 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaae6e2-b156-4df2-8c3b-b36a9ccf7d8a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:04.595110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaae6e2-b156-4df2-8c3b-b36a9ccf7d8a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:04.608101 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1008/2011] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:04 2022] PUT /identity/v3/projects/d493936e436640079d33418a420d671c/users/060e35e1df4940adb6b569b16fef779f/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:04.616302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab986b86-88eb-4459-be93-39fde5aed5e6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:04.619068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ab986b86-88eb-4459-be93-39fde5aed5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:04.667580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:04.680088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76526e-6c7d-4db1-944d-1d09ef9cdcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:04.680997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76526e-6c7d-4db1-944d-1d09ef9cdcb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:04.681292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76526e-6c7d-4db1-944d-1d09ef9cdcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:04.681576 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76526e-6c7d-4db1-944d-1d09ef9cdcb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:04.683199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76526e-6c7d-4db1-944d-1d09ef9cdcb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:04.683199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76526e-6c7d-4db1-944d-1d09ef9cdcb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:04.685230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76526e-6c7d-4db1-944d-1d09ef9cdcb0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:04.698153 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1004/2012] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:51:04.702685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76526e-6c7d-4db1-944d-1d09ef9cdcb0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:04.710523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab986b86-88eb-4459-be93-39fde5aed5e6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:04.714032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76526e-6c7d-4db1-944d-1d09ef9cdcb0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:04.716727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d409f-d451-45da-8c97-040f32e1da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:04.717142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d409f-d451-45da-8c97-040f32e1da67 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:04.717377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d409f-d451-45da-8c97-040f32e1da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:04.717623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d409f-d451-45da-8c97-040f32e1da67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:04.718363 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d409f-d451-45da-8c97-040f32e1da67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:04.718822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb3e91-9bd2-40eb-9b8d-e8e7038b62be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:04.719560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb3e91-9bd2-40eb-9b8d-e8e7038b62be admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:04.719779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb3e91-9bd2-40eb-9b8d-e8e7038b62be admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:04.719995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d409f-d451-45da-8c97-040f32e1da67 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:04.720318 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb3e91-9bd2-40eb-9b8d-e8e7038b62be admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:04.721609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb3e91-9bd2-40eb-9b8d-e8e7038b62be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:04.723890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb3e91-9bd2-40eb-9b8d-e8e7038b62be admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:04.731470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1009/2013] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:04.735752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d409f-d451-45da-8c97-040f32e1da67 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:04.746204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d409f-d451-45da-8c97-040f32e1da67 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:04.752627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e46c37-d01b-4480-9641-46ef8b7e04d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:04.753586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e46c37-d01b-4480-9641-46ef8b7e04d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:04.753867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e46c37-d01b-4480-9641-46ef8b7e04d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:04.754143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e46c37-d01b-4480-9641-46ef8b7e04d3 admin admin] PATH_INFO: `/v3/projects/d493936e436640079d33418a420d671c/users/060e35e1df4940adb6b569b16fef779f/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:04.756190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e46c37-d01b-4480-9641-46ef8b7e04d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d493936e436640079d33418a420d671c, user_id=060e35e1df4940adb6b569b16fef779f, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:04.759048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e46c37-d01b-4480-9641-46ef8b7e04d3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:04.771609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1010/2014] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:04 2022] PUT /identity/v3/projects/d493936e436640079d33418a420d671c/users/060e35e1df4940adb6b569b16fef779f/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:04.787088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3426265-18e7-4643-939c-50890f9aa49b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:04.787384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3426265-18e7-4643-939c-50890f9aa49b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:04.787713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3426265-18e7-4643-939c-50890f9aa49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:04.837724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1005/2015] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:04 2022] POST /identity/v3/projects => generated 389 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:04.841304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb3e91-9bd2-40eb-9b8d-e8e7038b62be admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:04.864846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f3426265-18e7-4643-939c-50890f9aa49b None None] MFA Rules not processed for user `060e35e1df4940adb6b569b16fef779f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:04.871894 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f3426265-18e7-4643-939c-50890f9aa49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:04.995822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb3e91-9bd2-40eb-9b8d-e8e7038b62be admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:05.000098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:05.004416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e4c4d-b019-42dd-a123-c715d77ede62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:05.005679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e4c4d-b019-42dd-a123-c715d77ede62 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:05.006796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e4c4d-b019-42dd-a123-c715d77ede62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:05.007427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e4c4d-b019-42dd-a123-c715d77ede62 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:05.011401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e4c4d-b019-42dd-a123-c715d77ede62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:05.012300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1011/2016] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:05.017346 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e4c4d-b019-42dd-a123-c715d77ede62 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:05.025332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3426265-18e7-4643-939c-50890f9aa49b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:05.026872 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f3426265-18e7-4643-939c-50890f9aa49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:05.097936 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1006/2017] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:04 2022] POST /identity/v3/users => generated 295 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:05.099692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e4c4d-b019-42dd-a123-c715d77ede62 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:05.101879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3426265-18e7-4643-939c-50890f9aa49b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:05.105968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ededd-271c-443e-b57c-40cb661505ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:05.106731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ededd-271c-443e-b57c-40cb661505ce service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:05.107026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ededd-271c-443e-b57c-40cb661505ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:05.107306 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ededd-271c-443e-b57c-40cb661505ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:05.110707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e4c4d-b019-42dd-a123-c715d77ede62 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:05.116867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94f635-2cb4-4a08-983a-3378287f7d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:05.117612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94f635-2cb4-4a08-983a-3378287f7d75 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:05.117900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94f635-2cb4-4a08-983a-3378287f7d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:05.118173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ededd-271c-443e-b57c-40cb661505ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:05.118295 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94f635-2cb4-4a08-983a-3378287f7d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:05.118922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94f635-2cb4-4a08-983a-3378287f7d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:05.119633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ededd-271c-443e-b57c-40cb661505ce service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:05.120808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94f635-2cb4-4a08-983a-3378287f7d75 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:05.132936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1012/2018] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:05.133213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1007/2019] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:05.137763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ededd-271c-443e-b57c-40cb661505ce service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:05.150756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ededd-271c-443e-b57c-40cb661505ce service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:05.157228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc4409-a2d4-490d-9cc1-fcf41eef820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:05.158018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc4409-a2d4-490d-9cc1-fcf41eef820f admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:05.158370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc4409-a2d4-490d-9cc1-fcf41eef820f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:05.158985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc4409-a2d4-490d-9cc1-fcf41eef820f admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:05.159580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc4409-a2d4-490d-9cc1-fcf41eef820f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:05.162537 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc4409-a2d4-490d-9cc1-fcf41eef820f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:05.182236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1013/2020] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:05.187327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94f635-2cb4-4a08-983a-3378287f7d75 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:05.199352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94f635-2cb4-4a08-983a-3378287f7d75 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:05.208351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011d761-7cae-4293-9bd4-54f63b570d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:05.209316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011d761-7cae-4293-9bd4-54f63b570d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:05.209601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011d761-7cae-4293-9bd4-54f63b570d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:05.209879 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011d761-7cae-4293-9bd4-54f63b570d13 admin admin] PATH_INFO: `/v3/projects/e53b8038b4b146f39e3a2f40f887ca2a/users/0666b17c009c42ca95fb3a7fbae3c29a/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:05.231956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011d761-7cae-4293-9bd4-54f63b570d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53b8038b4b146f39e3a2f40f887ca2a, user_id=0666b17c009c42ca95fb3a7fbae3c29a, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:05.234239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011d761-7cae-4293-9bd4-54f63b570d13 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:05.327725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc4409-a2d4-490d-9cc1-fcf41eef820f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:05.329681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-38dc4409-a2d4-490d-9cc1-fcf41eef820f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:05.345926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc4409-a2d4-490d-9cc1-fcf41eef820f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:05.346042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1008/2021] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:05 2022] PUT /identity/v3/projects/e53b8038b4b146f39e3a2f40f887ca2a/users/0666b17c009c42ca95fb3a7fbae3c29a/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:05.356125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c0f36-b3db-4359-835a-4dd2694971e7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:05.356541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c0f36-b3db-4359-835a-4dd2694971e7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:05.357151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c0f36-b3db-4359-835a-4dd2694971e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:05.388934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c91c0f36-b3db-4359-835a-4dd2694971e7 None None] MFA Rules not processed for user `0666b17c009c42ca95fb3a7fbae3c29a`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:05.390321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c91c0f36-b3db-4359-835a-4dd2694971e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:05.415502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85540c-4576-4da3-a5d5-9e0c8bb9dae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:05.416175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85540c-4576-4da3-a5d5-9e0c8bb9dae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:05.416465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85540c-4576-4da3-a5d5-9e0c8bb9dae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:05.416745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85540c-4576-4da3-a5d5-9e0c8bb9dae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:05.419158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cb85540c-4576-4da3-a5d5-9e0c8bb9dae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:05.505994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1009/2022] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:05.506567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85540c-4576-4da3-a5d5-9e0c8bb9dae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:05.507428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c0f36-b3db-4359-835a-4dd2694971e7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:05.508487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85540c-4576-4da3-a5d5-9e0c8bb9dae6 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:05.522456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:05.538192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1014/2023] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:05.603980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c0f36-b3db-4359-835a-4dd2694971e7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:05.608676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f8779-9672-4f57-b285-7378ccde5d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:05.609380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f8779-9672-4f57-b285-7378ccde5d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:05.609667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f8779-9672-4f57-b285-7378ccde5d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:05.609947 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f8779-9672-4f57-b285-7378ccde5d01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:05.610675 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f8779-9672-4f57-b285-7378ccde5d01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:05.612640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f8779-9672-4f57-b285-7378ccde5d01 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:05.761710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1010/2024] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:05 2022] POST /identity/v3/projects => generated 389 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:05.772841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85540c-4576-4da3-a5d5-9e0c8bb9dae6 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:05.795111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85540c-4576-4da3-a5d5-9e0c8bb9dae6 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:05.854079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:05.884854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699ae46-bb0c-4bbd-9749-167d1fdcc6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:05.885960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699ae46-bb0c-4bbd-9749-167d1fdcc6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:05.886462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699ae46-bb0c-4bbd-9749-167d1fdcc6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:05.886975 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699ae46-bb0c-4bbd-9749-167d1fdcc6e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:05.888271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699ae46-bb0c-4bbd-9749-167d1fdcc6e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:05.892167 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699ae46-bb0c-4bbd-9749-167d1fdcc6e7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:06.034681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1015/2025] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:05 2022] POST /identity/v3/users => generated 301 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:06.039328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f8779-9672-4f57-b285-7378ccde5d01 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:06.050940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f8779-9672-4f57-b285-7378ccde5d01 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:06.058383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f72dde-44f2-465a-af08-b1ff5eb49b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:06.059279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f72dde-44f2-465a-af08-b1ff5eb49b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:06.059720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f72dde-44f2-465a-af08-b1ff5eb49b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:06.061034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f72dde-44f2-465a-af08-b1ff5eb49b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:06.061034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f72dde-44f2-465a-af08-b1ff5eb49b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:06.068028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699ae46-bb0c-4bbd-9749-167d1fdcc6e7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:06.068235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f72dde-44f2-465a-af08-b1ff5eb49b99 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:06.075482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4699ae46-bb0c-4bbd-9749-167d1fdcc6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:06.095648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1011/2026] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:06.113856 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699ae46-bb0c-4bbd-9749-167d1fdcc6e7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:06.114521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f72dde-44f2-465a-af08-b1ff5eb49b99 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:06.129625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d8d08-6b07-4891-a906-2d0b10c30ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:06.130527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d8d08-6b07-4891-a906-2d0b10c30ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:06.130807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d8d08-6b07-4891-a906-2d0b10c30ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:06.131082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d8d08-6b07-4891-a906-2d0b10c30ce3 admin admin] PATH_INFO: `/v3/users/bf8704fab62845e0a4e40eecc2e52b76` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:06.135742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f72dde-44f2-465a-af08-b1ff5eb49b99 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:06.137255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d8d08-6b07-4891-a906-2d0b10c30ce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf8704fab62845e0a4e40eecc2e52b76)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:06.139340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d8d08-6b07-4891-a906-2d0b10c30ce3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:06.153784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cd8d7-ad26-41a3-82a3-edea43949da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:06.155558 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd8d7-ad26-41a3-82a3-edea43949da5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:06.156652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd8d7-ad26-41a3-82a3-edea43949da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:06.157441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd8d7-ad26-41a3-82a3-edea43949da5 admin admin] PATH_INFO: `/v3/projects/626f4b3d34f243a098e3cb81b19296ea/users/bfbedb00fdbd49b0bc56fe1ff69efbaa/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:06.192106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cd8d7-ad26-41a3-82a3-edea43949da5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=626f4b3d34f243a098e3cb81b19296ea, user_id=bfbedb00fdbd49b0bc56fe1ff69efbaa, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:06.196564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cd8d7-ad26-41a3-82a3-edea43949da5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:06.293482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1012/2027] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:06 2022] PUT /identity/v3/projects/626f4b3d34f243a098e3cb81b19296ea/users/bfbedb00fdbd49b0bc56fe1ff69efbaa/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:06.307380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cd8d7-ad26-41a3-82a3-edea43949da5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:06.421340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cd8d7-ad26-41a3-82a3-edea43949da5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:06.429123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e23d8d08-6b07-4891-a906-2d0b10c30ce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf8704fab62845e0a4e40eecc2e52b76'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:06.431427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a28324-66de-407e-8672-b3d54bad2ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:06.432582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a28324-66de-407e-8672-b3d54bad2ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:06.433536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a28324-66de-407e-8672-b3d54bad2ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:06.433959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a28324-66de-407e-8672-b3d54bad2ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:06.435006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a28324-66de-407e-8672-b3d54bad2ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:06.438059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a28324-66de-407e-8672-b3d54bad2ab0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:06.452827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1013/2028] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:06.456987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a28324-66de-407e-8672-b3d54bad2ab0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:06.480110 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e23d8d08-6b07-4891-a906-2d0b10c30ce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf8704fab62845e0a4e40eecc2e52b76'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:06.485954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a28324-66de-407e-8672-b3d54bad2ab0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:06.501658 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e23d8d08-6b07-4891-a906-2d0b10c30ce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf8704fab62845e0a4e40eecc2e52b76'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:06.507074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1135b-9291-4e62-9369-442d2950d2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:06.508140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1135b-9291-4e62-9369-442d2950d2fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:06.513728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1135b-9291-4e62-9369-442d2950d2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:06.513728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1135b-9291-4e62-9369-442d2950d2fa admin admin] PATH_INFO: `/v3/projects/626f4b3d34f243a098e3cb81b19296ea/users/bfbedb00fdbd49b0bc56fe1ff69efbaa/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:06.513728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1135b-9291-4e62-9369-442d2950d2fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=626f4b3d34f243a098e3cb81b19296ea, user_id=bfbedb00fdbd49b0bc56fe1ff69efbaa, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:06.513728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1135b-9291-4e62-9369-442d2950d2fa admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:06.527276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1014/2029] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:06 2022] PUT /identity/v3/projects/626f4b3d34f243a098e3cb81b19296ea/users/bfbedb00fdbd49b0bc56fe1ff69efbaa/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:06.538620 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c838-21e2-4abd-bd3a-fe8138186f60 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:06.538907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c838-21e2-4abd-bd3a-fe8138186f60 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:06.539274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c838-21e2-4abd-bd3a-fe8138186f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:06.571803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b2d8c838-21e2-4abd-bd3a-fe8138186f60 None None] MFA Rules not processed for user `bfbedb00fdbd49b0bc56fe1ff69efbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:06.573064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b2d8c838-21e2-4abd-bd3a-fe8138186f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:06.587104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e23d8d08-6b07-4891-a906-2d0b10c30ce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf8704fab62845e0a4e40eecc2e52b76'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:06.591145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e23d8d08-6b07-4891-a906-2d0b10c30ce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf8704fab62845e0a4e40eecc2e52b76'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:06.599787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1016/2030] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:06 2022] DELETE /identity/v3/users/bf8704fab62845e0a4e40eecc2e52b76 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:06.605119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:06.605472 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02dc4d-90a1-4b3c-9f8a-0b5c83cb70f1 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:06.605955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02dc4d-90a1-4b3c-9f8a-0b5c83cb70f1 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:06.606129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02dc4d-90a1-4b3c-9f8a-0b5c83cb70f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:06.632324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3b02dc4d-90a1-4b3c-9f8a-0b5c83cb70f1 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:06.633610 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3b02dc4d-90a1-4b3c-9f8a-0b5c83cb70f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:06.656858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1015/2031] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:06.661584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8c838-21e2-4abd-bd3a-fe8138186f60 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:06.662980 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b2d8c838-21e2-4abd-bd3a-fe8138186f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:06.721206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1017/2032] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:06.721386 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02dc4d-90a1-4b3c-9f8a-0b5c83cb70f1 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:06.722713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3b02dc4d-90a1-4b3c-9f8a-0b5c83cb70f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:06.751323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8c838-21e2-4abd-bd3a-fe8138186f60 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:06.757335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dbf31-595f-450d-ae66-0bbed2764c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:06.758825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dbf31-595f-450d-ae66-0bbed2764c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:06.759870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dbf31-595f-450d-ae66-0bbed2764c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:06.760155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dbf31-595f-450d-ae66-0bbed2764c98 admin admin] PATH_INFO: `/v3/users/4ab8aeaeb8364d359b9e28386e78e19e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:06.761550 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90dbf31-595f-450d-ae66-0bbed2764c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab8aeaeb8364d359b9e28386e78e19e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:06.764129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90dbf31-595f-450d-ae66-0bbed2764c98 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:06.777240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:06.800617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02dc4d-90a1-4b3c-9f8a-0b5c83cb70f1 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:06.804581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38029e-a461-41e6-bf98-bbc743be35a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:06.805284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38029e-a461-41e6-bf98-bbc743be35a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:06.805574 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38029e-a461-41e6-bf98-bbc743be35a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:06.805877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38029e-a461-41e6-bf98-bbc743be35a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:06.874454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38029e-a461-41e6-bf98-bbc743be35a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:06.875828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38029e-a461-41e6-bf98-bbc743be35a0 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:06.885752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1018/2033] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:06.888705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38029e-a461-41e6-bf98-bbc743be35a0 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:06.890161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dd38029e-a461-41e6-bf98-bbc743be35a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:06.920960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a90dbf31-595f-450d-ae66-0bbed2764c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab8aeaeb8364d359b9e28386e78e19e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:06.937804 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a90dbf31-595f-450d-ae66-0bbed2764c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab8aeaeb8364d359b9e28386e78e19e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:06.967660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a90dbf31-595f-450d-ae66-0bbed2764c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab8aeaeb8364d359b9e28386e78e19e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:06.996695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38029e-a461-41e6-bf98-bbc743be35a0 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:07.005887 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96d69-951d-4645-a9a5-8dde3f2d9d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:07.006564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96d69-951d-4645-a9a5-8dde3f2d9d8a service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:07.006886 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96d69-951d-4645-a9a5-8dde3f2d9d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:07.007138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96d69-951d-4645-a9a5-8dde3f2d9d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:07.019292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c96d69-951d-4645-a9a5-8dde3f2d9d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:07.020804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c96d69-951d-4645-a9a5-8dde3f2d9d8a service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:07.035255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1019/2034] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:07.055611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96d69-951d-4645-a9a5-8dde3f2d9d8a service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:07.057337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d2c96d69-951d-4645-a9a5-8dde3f2d9d8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:07.071365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96d69-951d-4645-a9a5-8dde3f2d9d8a service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:07.078506 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:07.079886 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:07.080227 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:07.080564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] PATH_INFO: `/v3/users/0666b17c009c42ca95fb3a7fbae3c29a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:07.082396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0666b17c009c42ca95fb3a7fbae3c29a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:07.084911 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:07.104182 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a90dbf31-595f-450d-ae66-0bbed2764c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab8aeaeb8364d359b9e28386e78e19e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.105301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a90dbf31-595f-450d-ae66-0bbed2764c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab8aeaeb8364d359b9e28386e78e19e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.115742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1016/2035] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:06 2022] DELETE /identity/v3/users/4ab8aeaeb8364d359b9e28386e78e19e => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:07.118686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872494a-9ab9-465b-8d76-5001db223944 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:07.118686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872494a-9ab9-465b-8d76-5001db223944 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:07.119007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872494a-9ab9-465b-8d76-5001db223944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:07.151742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7872494a-9ab9-465b-8d76-5001db223944 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:07.151742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7872494a-9ab9-465b-8d76-5001db223944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:07.207135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:07.227316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1017/2036] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:07.245020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872494a-9ab9-465b-8d76-5001db223944 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:07.246575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7872494a-9ab9-465b-8d76-5001db223944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:07.319871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872494a-9ab9-465b-8d76-5001db223944 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:07.325598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d71bfc-4988-43b5-b839-62f9fe95b3b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:07.326402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d71bfc-4988-43b5-b839-62f9fe95b3b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:07.326830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d71bfc-4988-43b5-b839-62f9fe95b3b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:07.327178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d71bfc-4988-43b5-b839-62f9fe95b3b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:07.339684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d71bfc-4988-43b5-b839-62f9fe95b3b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:07.341430 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d71bfc-4988-43b5-b839-62f9fe95b3b4 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:07.342018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0666b17c009c42ca95fb3a7fbae3c29a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.358421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1018/2037] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:07.365831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0666b17c009c42ca95fb3a7fbae3c29a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.380133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0666b17c009c42ca95fb3a7fbae3c29a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.491267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0666b17c009c42ca95fb3a7fbae3c29a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.493293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0666b17c009c42ca95fb3a7fbae3c29a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.500267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1020/2038] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:07 2022] DELETE /identity/v3/users/0666b17c009c42ca95fb3a7fbae3c29a => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:07.506151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4232e08-34db-40dc-9173-c164f410f5f3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:07.506447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4232e08-34db-40dc-9173-c164f410f5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:07.506726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4232e08-34db-40dc-9173-c164f410f5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:07.556832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b4232e08-34db-40dc-9173-c164f410f5f3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:07.558644 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b4232e08-34db-40dc-9173-c164f410f5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:07.642660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1019/2039] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:07.657960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:07.661070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:07.669790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:07.709830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4232e08-34db-40dc-9173-c164f410f5f3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:07.711897 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b4232e08-34db-40dc-9173-c164f410f5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:07.731010 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ebf76-5363-4bc3-9068-2fd70a204067 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:07.731201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4232e08-34db-40dc-9173-c164f410f5f3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:07.736909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:07.738006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:07.738097 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353d9c6-e5d6-46ff-8766-6170b77ff81e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:07.738281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:07.738518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] PATH_INFO: `/v3/projects/91ee9be2fdb44236911f0636b5b6b4e0` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:07.738732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353d9c6-e5d6-46ff-8766-6170b77ff81e service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:07.739528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353d9c6-e5d6-46ff-8766-6170b77ff81e service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:07.739834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353d9c6-e5d6-46ff-8766-6170b77ff81e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:07.740491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ee9be2fdb44236911f0636b5b6b4e0)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:07.742872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:07.752613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353d9c6-e5d6-46ff-8766-6170b77ff81e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:07.754223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353d9c6-e5d6-46ff-8766-6170b77ff81e service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:07.769390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1021/2040] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:07.891292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ee9be2fdb44236911f0636b5b6b4e0', 'request_id': 'req-d6df92dc-bb9b-4a70-b3de-14c070503367'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.899989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ee9be2fdb44236911f0636b5b6b4e0', 'request_id': 'req-d6df92dc-bb9b-4a70-b3de-14c070503367'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.910178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ee9be2fdb44236911f0636b5b6b4e0', 'request_id': 'req-d6df92dc-bb9b-4a70-b3de-14c070503367'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.982198 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] The token cache is being invalidate because project 91ee9be2fdb44236911f0636b5b6b4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:07.982391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.983539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:07.985358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1020/2041] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:07 2022] DELETE /identity/v3/projects/91ee9be2fdb44236911f0636b5b6b4e0 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:07.989305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353d9c6-e5d6-46ff-8766-6170b77ff81e service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:08.001480 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353d9c6-e5d6-46ff-8766-6170b77ff81e service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:08.080739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:08.081707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:08.081984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:08.082249 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] PATH_INFO: `/v3/users/fc25255b8ab549929d8b5ecb79253b83` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:08.084673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc25255b8ab549929d8b5ecb79253b83)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:08.086702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:08.102139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:08.103661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:08.149209 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df92dc-bb9b-4a70-b3de-14c070503367 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:08.154984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:08.155877 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:08.156253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:08.156762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] PATH_INFO: `/v3/projects/0b8636400dd840e7b5b6dceb7c31d5f5` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:08.159019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8636400dd840e7b5b6dceb7c31d5f5)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:08.162046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:08.170134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:08.340462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc25255b8ab549929d8b5ecb79253b83'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.340722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8636400dd840e7b5b6dceb7c31d5f5', 'request_id': 'req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.351142 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8636400dd840e7b5b6dceb7c31d5f5', 'request_id': 'req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.358416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8636400dd840e7b5b6dceb7c31d5f5', 'request_id': 'req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.362305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc25255b8ab549929d8b5ecb79253b83'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.373839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc25255b8ab549929d8b5ecb79253b83'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.438514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] The token cache is being invalidate because project 0b8636400dd840e7b5b6dceb7c31d5f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:08.438815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.439629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.441335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1021/2042] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:08 2022] DELETE /identity/v3/projects/0b8636400dd840e7b5b6dceb7c31d5f5 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:08.442111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:08.443635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:08.484072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc25255b8ab549929d8b5ecb79253b83'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.484749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc25255b8ab549929d8b5ecb79253b83'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.495215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1022/2043] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:07 2022] DELETE /identity/v3/users/fc25255b8ab549929d8b5ecb79253b83 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:08.496142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:08.497550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:08.523593 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ded11f-ca7d-4c4c-bbdf-7836ef7d5a76 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:08.546961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:08.594902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b1de5-ddd9-435f-b211-dfef490bc8cc admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:08.605700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:08.606682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:08.606887 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:08.607158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] PATH_INFO: `/v3/users/060e35e1df4940adb6b569b16fef779f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:08.608488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=060e35e1df4940adb6b569b16fef779f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:08.610481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:08.614351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:08.615245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:08.615517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:08.615800 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] PATH_INFO: `/v3/projects/e53b8038b4b146f39e3a2f40f887ca2a` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:08.617683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e53b8038b4b146f39e3a2f40f887ca2a)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:08.620567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:08.751338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e53b8038b4b146f39e3a2f40f887ca2a', 'request_id': 'req-6546c3fa-90a6-4862-8b0b-93875731a8ae'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.759152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e53b8038b4b146f39e3a2f40f887ca2a', 'request_id': 'req-6546c3fa-90a6-4862-8b0b-93875731a8ae'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.768465 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e53b8038b4b146f39e3a2f40f887ca2a', 'request_id': 'req-6546c3fa-90a6-4862-8b0b-93875731a8ae'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.800019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:08.837014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] The token cache is being invalidate because project e53b8038b4b146f39e3a2f40f887ca2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:08.838546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.839279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.841598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1022/2044] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:08 2022] DELETE /identity/v3/projects/e53b8038b4b146f39e3a2f40f887ca2a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:08.845378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:08.848287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:08.855944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '060e35e1df4940adb6b569b16fef779f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.867360 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '060e35e1df4940adb6b569b16fef779f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.874818 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '060e35e1df4940adb6b569b16fef779f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.923269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546c3fa-90a6-4862-8b0b-93875731a8ae admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:08.928110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882ab64-5269-482f-a3fd-501b9482d644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:08.929047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882ab64-5269-482f-a3fd-501b9482d644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:08.929333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882ab64-5269-482f-a3fd-501b9482d644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:08.929603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882ab64-5269-482f-a3fd-501b9482d644 admin admin] PATH_INFO: `/v3/users/bfbedb00fdbd49b0bc56fe1ff69efbaa` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:08.930897 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882ab64-5269-482f-a3fd-501b9482d644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfbedb00fdbd49b0bc56fe1ff69efbaa)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:08.933032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d882ab64-5269-482f-a3fd-501b9482d644 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:08.987087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '060e35e1df4940adb6b569b16fef779f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.987800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '060e35e1df4940adb6b569b16fef779f'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:08.995089 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1023/2045] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:08 2022] DELETE /identity/v3/users/060e35e1df4940adb6b569b16fef779f => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:09.075926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:09.077667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:09.153560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d882ab64-5269-482f-a3fd-501b9482d644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfbedb00fdbd49b0bc56fe1ff69efbaa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.167208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e758ee6-db21-48b2-a3dc-ba4107980d10 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:09.172642 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:09.173583 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:09.173874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:09.174300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] PATH_INFO: `/v3/projects/65800b8d6d494258be979728d4b9fdaf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:09.175434 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65800b8d6d494258be979728d4b9fdaf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:09.177410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:09.181702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d882ab64-5269-482f-a3fd-501b9482d644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfbedb00fdbd49b0bc56fe1ff69efbaa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.198335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d882ab64-5269-482f-a3fd-501b9482d644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfbedb00fdbd49b0bc56fe1ff69efbaa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.262145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d882ab64-5269-482f-a3fd-501b9482d644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfbedb00fdbd49b0bc56fe1ff69efbaa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.262801 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d882ab64-5269-482f-a3fd-501b9482d644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfbedb00fdbd49b0bc56fe1ff69efbaa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.281044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1023/2046] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:08 2022] DELETE /identity/v3/users/bfbedb00fdbd49b0bc56fe1ff69efbaa => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:09.325565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65800b8d6d494258be979728d4b9fdaf', 'request_id': 'req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.416016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65800b8d6d494258be979728d4b9fdaf', 'request_id': 'req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.421602 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65800b8d6d494258be979728d4b9fdaf', 'request_id': 'req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.436345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] The token cache is being invalidate because project 65800b8d6d494258be979728d4b9fdaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:09.436747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.437680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.439417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1024/2047] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:09 2022] DELETE /identity/v3/projects/65800b8d6d494258be979728d4b9fdaf => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:09.445016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46de7-4896-4ab4-8be0-d5dc73caeb3f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:09.445368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46de7-4896-4ab4-8be0-d5dc73caeb3f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:09.445649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b46de7-4896-4ab4-8be0-d5dc73caeb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:09.462133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:09.466880 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-28b46de7-4896-4ab4-8be0-d5dc73caeb3f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:09.468180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-28b46de7-4896-4ab4-8be0-d5dc73caeb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:09.502047 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:09.503814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:09.566394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1024/2048] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:09.578074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d938c3-7d9b-4753-a847-2800bfdf76a9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:09.578380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d938c3-7d9b-4753-a847-2800bfdf76a9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:09.578649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d938c3-7d9b-4753-a847-2800bfdf76a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:09.633713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-32d938c3-7d9b-4753-a847-2800bfdf76a9 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:09.635251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-32d938c3-7d9b-4753-a847-2800bfdf76a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:09.651507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1025/2049] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:09.657103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d938c3-7d9b-4753-a847-2800bfdf76a9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:09.664864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b4563-3ac8-47f0-abc0-b8e5b0b64ae9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:09.669280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d938c3-7d9b-4753-a847-2800bfdf76a9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:09.674962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2c9b7-8b24-48a6-980c-b3409678880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:09.677097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2c9b7-8b24-48a6-980c-b3409678880f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:09.677455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2c9b7-8b24-48a6-980c-b3409678880f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:09.677792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2c9b7-8b24-48a6-980c-b3409678880f admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:09.678762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2c9b7-8b24-48a6-980c-b3409678880f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:09.679148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2c9b7-8b24-48a6-980c-b3409678880f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:09.681560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb2c9b7-8b24-48a6-980c-b3409678880f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:09.682175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:09.684012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:09.684398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:09.684733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] PATH_INFO: `/v3/projects/d493936e436640079d33418a420d671c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:09.687958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d493936e436640079d33418a420d671c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:09.691644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:09.698836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1026/2050] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:09.705308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2c9b7-8b24-48a6-980c-b3409678880f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:09.708570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4cb2c9b7-8b24-48a6-980c-b3409678880f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:09.725492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb2c9b7-8b24-48a6-980c-b3409678880f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:09.731496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:09.732248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:09.732586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:09.733425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] PATH_INFO: `/v3/users/30c0536cbbd94332a3e5a1eaf4405e0d` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:09.734385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c0536cbbd94332a3e5a1eaf4405e0d)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:09.737004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:09.822903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd493936e436640079d33418a420d671c', 'request_id': 'req-748f335c-4048-471d-8270-7bc66f7ba798'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.965168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd493936e436640079d33418a420d671c', 'request_id': 'req-748f335c-4048-471d-8270-7bc66f7ba798'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.981504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd493936e436640079d33418a420d671c', 'request_id': 'req-748f335c-4048-471d-8270-7bc66f7ba798'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:09.999567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c0536cbbd94332a3e5a1eaf4405e0d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.012677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] The token cache is being invalidate because project d493936e436640079d33418a420d671c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:10.014415 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.017065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.022777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1025/2051] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:09 2022] DELETE /identity/v3/projects/d493936e436640079d33418a420d671c => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:10.022777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.023372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:10.024248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c0536cbbd94332a3e5a1eaf4405e0d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.047362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c0536cbbd94332a3e5a1eaf4405e0d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.106517 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f335c-4048-471d-8270-7bc66f7ba798 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.111765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3c7b8-2cdc-45fe-9cce-6932b3bb24ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.112543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3c7b8-2cdc-45fe-9cce-6932b3bb24ec admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.112909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3c7b8-2cdc-45fe-9cce-6932b3bb24ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.113257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3c7b8-2cdc-45fe-9cce-6932b3bb24ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.113607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c0536cbbd94332a3e5a1eaf4405e0d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.114749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3c7b8-2cdc-45fe-9cce-6932b3bb24ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.115813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:10.116783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c0536cbbd94332a3e5a1eaf4405e0d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.116928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3c7b8-2cdc-45fe-9cce-6932b3bb24ec admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.124918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1027/2052] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:09 2022] DELETE /identity/v3/users/30c0536cbbd94332a3e5a1eaf4405e0d => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:10.126059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.127541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:10.192242 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda8075-ea9c-4abe-b6af-d8d4a6c799fe admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.198233 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.198651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.198936 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.199220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] PATH_INFO: `/v3/projects/626f4b3d34f243a098e3cb81b19296ea` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.200414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=626f4b3d34f243a098e3cb81b19296ea)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.204078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1026/2053] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:10 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:10.204376 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.204478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da0069-c82d-4360-bb40-34165bd35726 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.204478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da0069-c82d-4360-bb40-34165bd35726 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.204478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da0069-c82d-4360-bb40-34165bd35726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.226716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-61da0069-c82d-4360-bb40-34165bd35726 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:10.226926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-61da0069-c82d-4360-bb40-34165bd35726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:10.292489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1027/2054] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:10.293192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567c656-6005-4df9-a309-a7d89d92b7ad None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.293548 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567c656-6005-4df9-a309-a7d89d92b7ad None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.293781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567c656-6005-4df9-a309-a7d89d92b7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.317349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c567c656-6005-4df9-a309-a7d89d92b7ad None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:10.317349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c567c656-6005-4df9-a309-a7d89d92b7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:10.336827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '626f4b3d34f243a098e3cb81b19296ea', 'request_id': 'req-8da20580-ad08-43b2-9967-337131945bdc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.345925 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '626f4b3d34f243a098e3cb81b19296ea', 'request_id': 'req-8da20580-ad08-43b2-9967-337131945bdc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.353440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '626f4b3d34f243a098e3cb81b19296ea', 'request_id': 'req-8da20580-ad08-43b2-9967-337131945bdc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.409008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1028/2055] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:10.410046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567c656-6005-4df9-a309-a7d89d92b7ad None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.412626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c567c656-6005-4df9-a309-a7d89d92b7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:10.426018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567c656-6005-4df9-a309-a7d89d92b7ad None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.431810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17adff71-1a47-4c90-91eb-1402587ce6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.432599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adff71-1a47-4c90-91eb-1402587ce6ac admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.432919 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adff71-1a47-4c90-91eb-1402587ce6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.433214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17adff71-1a47-4c90-91eb-1402587ce6ac admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.440493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adff71-1a47-4c90-91eb-1402587ce6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.440615 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] The token cache is being invalidate because project 626f4b3d34f243a098e3cb81b19296ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:10.440615 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.440615 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8da20580-ad08-43b2-9967-337131945bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:10.440839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17adff71-1a47-4c90-91eb-1402587ce6ac admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.440955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1028/2056] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:10 2022] DELETE /identity/v3/projects/626f4b3d34f243a098e3cb81b19296ea => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:10.440955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaa46a-61eb-4b2c-af9a-984780a6b3af None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.440955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaa46a-61eb-4b2c-af9a-984780a6b3af None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.440955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebaa46a-61eb-4b2c-af9a-984780a6b3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.458999 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8ebaa46a-61eb-4b2c-af9a-984780a6b3af None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:10.461383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8ebaa46a-61eb-4b2c-af9a-984780a6b3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:10.468944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1029/2057] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:10.469863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaa46a-61eb-4b2c-af9a-984780a6b3af None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.471262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8ebaa46a-61eb-4b2c-af9a-984780a6b3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:10.507807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1029/2058] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:10 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:10.509390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0478eb3-2935-4ce8-a87f-3e164400db92 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.509755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0478eb3-2935-4ce8-a87f-3e164400db92 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.510139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0478eb3-2935-4ce8-a87f-3e164400db92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.528832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a0478eb3-2935-4ce8-a87f-3e164400db92 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:10.530204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a0478eb3-2935-4ce8-a87f-3e164400db92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:10.537774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebaa46a-61eb-4b2c-af9a-984780a6b3af None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.539023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1030/2059] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:10.539829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0478eb3-2935-4ce8-a87f-3e164400db92 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.543471 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dea82-a7a3-440e-9ca7-4969cda02215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.544482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dea82-a7a3-440e-9ca7-4969cda02215 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.544747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dea82-a7a3-440e-9ca7-4969cda02215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.545139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dea82-a7a3-440e-9ca7-4969cda02215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.547500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4d1dea82-a7a3-440e-9ca7-4969cda02215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:10.549952 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0478eb3-2935-4ce8-a87f-3e164400db92 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.554897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0d396-6216-416e-874e-ae87bc911730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.555562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0d396-6216-416e-874e-ae87bc911730 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.555880 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0d396-6216-416e-874e-ae87bc911730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.556124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0d396-6216-416e-874e-ae87bc911730 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.556966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0d396-6216-416e-874e-ae87bc911730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:10.557278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0d396-6216-416e-874e-ae87bc911730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.558897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0d396-6216-416e-874e-ae87bc911730 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.560827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1dea82-a7a3-440e-9ca7-4969cda02215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.562294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1dea82-a7a3-440e-9ca7-4969cda02215 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.574529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1031/2060] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:10.574529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0d396-6216-416e-874e-ae87bc911730 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.574795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1030/2061] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:10.574873 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8aa0d396-6216-416e-874e-ae87bc911730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:10.574946 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c381f18-e198-403a-b7dc-0361a1780950 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.574946 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c381f18-e198-403a-b7dc-0361a1780950 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.574946 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c381f18-e198-403a-b7dc-0361a1780950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.587678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0d396-6216-416e-874e-ae87bc911730 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.593612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a11fc7-c3cd-403e-af7d-69bee18cf0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.594298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a11fc7-c3cd-403e-af7d-69bee18cf0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.595003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a11fc7-c3cd-403e-af7d-69bee18cf0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.595003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a11fc7-c3cd-403e-af7d-69bee18cf0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.595592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a11fc7-c3cd-403e-af7d-69bee18cf0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.595940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5c381f18-e198-403a-b7dc-0361a1780950 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:10.597219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5c381f18-e198-403a-b7dc-0361a1780950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:10.597349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a11fc7-c3cd-403e-af7d-69bee18cf0e8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.601352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:10.607043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1031/2062] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:10.611537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c381f18-e198-403a-b7dc-0361a1780950 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.618931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1032/2063] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:10.620945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a11fc7-c3cd-403e-af7d-69bee18cf0e8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.630200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c381f18-e198-403a-b7dc-0361a1780950 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.647355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a11fc7-c3cd-403e-af7d-69bee18cf0e8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.654245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1633de-435c-4202-9393-577c30270270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.654245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1633de-435c-4202-9393-577c30270270 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.654245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1633de-435c-4202-9393-577c30270270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.654245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1633de-435c-4202-9393-577c30270270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.654790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1633de-435c-4202-9393-577c30270270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.656914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e6c48-971b-4c71-9df7-30162775a4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.658531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e6c48-971b-4c71-9df7-30162775a4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.662897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e6c48-971b-4c71-9df7-30162775a4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.662897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e6c48-971b-4c71-9df7-30162775a4bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.662897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e6c48-971b-4c71-9df7-30162775a4bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:10.662897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e6c48-971b-4c71-9df7-30162775a4bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.662897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e6c48-971b-4c71-9df7-30162775a4bf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.671016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1633de-435c-4202-9393-577c30270270 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.682479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1033/2064] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:10.683535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e6c48-971b-4c71-9df7-30162775a4bf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.697736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e6c48-971b-4c71-9df7-30162775a4bf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.704928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf4ce1-1e62-40db-b138-02d29b1e9d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.705813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf4ce1-1e62-40db-b138-02d29b1e9d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.706284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf4ce1-1e62-40db-b138-02d29b1e9d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.706570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf4ce1-1e62-40db-b138-02d29b1e9d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.707320 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf4ce1-1e62-40db-b138-02d29b1e9d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.714317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf4ce1-1e62-40db-b138-02d29b1e9d99 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.722931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1034/2065] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:10.724258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf4ce1-1e62-40db-b138-02d29b1e9d99 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.738199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf4ce1-1e62-40db-b138-02d29b1e9d99 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.747162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187517e4-e7da-4b7a-9ef6-09a180823a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:10.749034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187517e4-e7da-4b7a-9ef6-09a180823a1f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:10.749034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187517e4-e7da-4b7a-9ef6-09a180823a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:10.749034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187517e4-e7da-4b7a-9ef6-09a180823a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:10.749238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187517e4-e7da-4b7a-9ef6-09a180823a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:10.751020 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187517e4-e7da-4b7a-9ef6-09a180823a1f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:10.807922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1032/2066] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:10 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:10.809124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1633de-435c-4202-9393-577c30270270 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.837720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1035/2067] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:10 2022] POST /identity/v3/projects => generated 407 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:10.842197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187517e4-e7da-4b7a-9ef6-09a180823a1f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:10.901091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1633de-435c-4202-9393-577c30270270 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:10.997847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187517e4-e7da-4b7a-9ef6-09a180823a1f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:11.011227 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c102aa6-2ec3-4645-9044-a5d561ccce84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.012654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c102aa6-2ec3-4645-9044-a5d561ccce84 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.013292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c102aa6-2ec3-4645-9044-a5d561ccce84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.013780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c102aa6-2ec3-4645-9044-a5d561ccce84 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.015141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c102aa6-2ec3-4645-9044-a5d561ccce84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.018995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c102aa6-2ec3-4645-9044-a5d561ccce84 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.042352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a36b2d-03e5-4436-8ecf-3af0852ca455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.043578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a36b2d-03e5-4436-8ecf-3af0852ca455 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.043931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a36b2d-03e5-4436-8ecf-3af0852ca455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.044412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a36b2d-03e5-4436-8ecf-3af0852ca455 admin admin] PATH_INFO: `/v3/projects/2a8580d1c8784e78aee4f19aa83d0823/users/8617eb9fff41436d867e205da136d61f/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.074103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a36b2d-03e5-4436-8ecf-3af0852ca455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8580d1c8784e78aee4f19aa83d0823, user_id=8617eb9fff41436d867e205da136d61f, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.077852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a36b2d-03e5-4436-8ecf-3af0852ca455 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.111443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1036/2068] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:10 2022] POST /identity/v3/users => generated 292 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:11.113950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c102aa6-2ec3-4645-9044-a5d561ccce84 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:11.127609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c102aa6-2ec3-4645-9044-a5d561ccce84 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:11.131442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:11.138103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aac1e4-9a8a-494d-a883-e1c245f0ba35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.139354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aac1e4-9a8a-494d-a883-e1c245f0ba35 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.139794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aac1e4-9a8a-494d-a883-e1c245f0ba35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.142990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aac1e4-9a8a-494d-a883-e1c245f0ba35 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.142990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aac1e4-9a8a-494d-a883-e1c245f0ba35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.143328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aac1e4-9a8a-494d-a883-e1c245f0ba35 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.195238 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1033/2069] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:10 2022] PUT /identity/v3/projects/2a8580d1c8784e78aee4f19aa83d0823/users/8617eb9fff41436d867e205da136d61f/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:11.196025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a36b2d-03e5-4436-8ecf-3af0852ca455 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:11.230093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1037/2070] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:11 2022] POST /identity/v3/users => generated 304 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:11.231704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a544f-12da-4638-aaa1-3e7456a9074d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.232132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a544f-12da-4638-aaa1-3e7456a9074d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.232482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a544f-12da-4638-aaa1-3e7456a9074d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.260813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a97a544f-12da-4638-aaa1-3e7456a9074d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:11.260813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a97a544f-12da-4638-aaa1-3e7456a9074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:11.333969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a36b2d-03e5-4436-8ecf-3af0852ca455 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:11.333969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1a1a-006a-4603-8910-2db6cfa3ab94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.333969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1a1a-006a-4603-8910-2db6cfa3ab94 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.333969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1a1a-006a-4603-8910-2db6cfa3ab94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.343601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec1a1a-006a-4603-8910-2db6cfa3ab94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.343601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec1a1a-006a-4603-8910-2db6cfa3ab94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.343601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec1a1a-006a-4603-8910-2db6cfa3ab94 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.344033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1038/2071] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:11.344033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a544f-12da-4638-aaa1-3e7456a9074d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:11.344033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a97a544f-12da-4638-aaa1-3e7456a9074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:11.357709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a544f-12da-4638-aaa1-3e7456a9074d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:11.363414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1034/2072] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:11.365885 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1a1a-006a-4603-8910-2db6cfa3ab94 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:11.366546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.367442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.372569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.372569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] PATH_INFO: `/v3/projects/c445168cdf3f447eb9072e3f6da253d4` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.372569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c445168cdf3f447eb9072e3f6da253d4)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.372569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.377777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec1a1a-006a-4603-8910-2db6cfa3ab94 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:11.385211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd616ee-9620-4189-955e-8ff048937849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.385954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd616ee-9620-4189-955e-8ff048937849 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.386240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd616ee-9620-4189-955e-8ff048937849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.386893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd616ee-9620-4189-955e-8ff048937849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.387686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd616ee-9620-4189-955e-8ff048937849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.389471 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd616ee-9620-4189-955e-8ff048937849 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.404423 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1035/2073] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:11.408152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd616ee-9620-4189-955e-8ff048937849 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:11.410884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1dd616ee-9620-4189-955e-8ff048937849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:11.480447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd616ee-9620-4189-955e-8ff048937849 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:11.485724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7b374-bf17-4b72-a237-2696cabfc2e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.486391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7b374-bf17-4b72-a237-2696cabfc2e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.486685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7b374-bf17-4b72-a237-2696cabfc2e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.486969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7b374-bf17-4b72-a237-2696cabfc2e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.501669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7b374-bf17-4b72-a237-2696cabfc2e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.503509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7b374-bf17-4b72-a237-2696cabfc2e8 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.517779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c445168cdf3f447eb9072e3f6da253d4', 'request_id': 'req-7e515c8d-55ab-4278-9b12-c20802159169'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:11.518645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1036/2074] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:11.519515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7b374-bf17-4b72-a237-2696cabfc2e8 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:11.531812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7b374-bf17-4b72-a237-2696cabfc2e8 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:11.549046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:11.598920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c445168cdf3f447eb9072e3f6da253d4', 'request_id': 'req-7e515c8d-55ab-4278-9b12-c20802159169'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:11.599301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccd9ea-a7e3-455d-8e42-26222944422a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.599987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccd9ea-a7e3-455d-8e42-26222944422a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.600275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccd9ea-a7e3-455d-8e42-26222944422a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.600555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccd9ea-a7e3-455d-8e42-26222944422a admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.601340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ccd9ea-a7e3-455d-8e42-26222944422a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.602967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ccd9ea-a7e3-455d-8e42-26222944422a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.610314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c445168cdf3f447eb9072e3f6da253d4', 'request_id': 'req-7e515c8d-55ab-4278-9b12-c20802159169'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:11.621767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1037/2075] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:11.621767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccd9ea-a7e3-455d-8e42-26222944422a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:11.634665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] The token cache is being invalidate because project c445168cdf3f447eb9072e3f6da253d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:11.636598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:11.636598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:11.637529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccd9ea-a7e3-455d-8e42-26222944422a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:11.637651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1039/2076] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:11 2022] DELETE /identity/v3/projects/c445168cdf3f447eb9072e3f6da253d4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:11.641475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:11.643031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:11.647044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486bb37-d673-4472-bde3-eabea2e16341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.647723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486bb37-d673-4472-bde3-eabea2e16341 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.647987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486bb37-d673-4472-bde3-eabea2e16341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.648371 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486bb37-d673-4472-bde3-eabea2e16341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.649247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486bb37-d673-4472-bde3-eabea2e16341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.650957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486bb37-d673-4472-bde3-eabea2e16341 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.657650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e515c8d-55ab-4278-9b12-c20802159169 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:11.664669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1038/2077] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:11.671115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8458652-4df2-471d-8bb0-5589d6ddc463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.671436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486bb37-d673-4472-bde3-eabea2e16341 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:11.671693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8458652-4df2-471d-8bb0-5589d6ddc463 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.671693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8458652-4df2-471d-8bb0-5589d6ddc463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.671693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8458652-4df2-471d-8bb0-5589d6ddc463 admin admin] PATH_INFO: `/v3/projects/3859defa71b445aca9597bbeba642819/users/9431cd1afee04d408bceeb353b263241/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.671895 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d486bb37-d673-4472-bde3-eabea2e16341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:11.685204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486bb37-d673-4472-bde3-eabea2e16341 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:11.689972 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8458652-4df2-471d-8bb0-5589d6ddc463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3859defa71b445aca9597bbeba642819, user_id=9431cd1afee04d408bceeb353b263241, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.692973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8458652-4df2-471d-8bb0-5589d6ddc463 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.699903 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd28ddf-7864-456c-99b2-b584db50ca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.701123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd28ddf-7864-456c-99b2-b584db50ca32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.701642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd28ddf-7864-456c-99b2-b584db50ca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.702255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd28ddf-7864-456c-99b2-b584db50ca32 admin admin] PATH_INFO: `/v3/users/e1d118a5d4e6487f90744ecf58271823` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.703683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd28ddf-7864-456c-99b2-b584db50ca32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d118a5d4e6487f90744ecf58271823)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.705766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd28ddf-7864-456c-99b2-b584db50ca32 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.801460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1040/2078] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:11 2022] PUT /identity/v3/projects/3859defa71b445aca9597bbeba642819/users/9431cd1afee04d408bceeb353b263241/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:11.801460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8458652-4df2-471d-8bb0-5589d6ddc463 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:11.801460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b8458652-4df2-471d-8bb0-5589d6ddc463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:11.875758 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8458652-4df2-471d-8bb0-5589d6ddc463 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:11.881907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342791b1-3198-4c08-bd1d-8116bceadde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:11.882809 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342791b1-3198-4c08-bd1d-8116bceadde4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:11.883095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342791b1-3198-4c08-bd1d-8116bceadde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:11.883383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342791b1-3198-4c08-bd1d-8116bceadde4 admin admin] PATH_INFO: `/v3/projects/d62c01d6a8ad4051b60c4a95a1ce13ae/users/bd11db69638945b087b842059d7e47c0/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:11.899014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342791b1-3198-4c08-bd1d-8116bceadde4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62c01d6a8ad4051b60c4a95a1ce13ae, user_id=bd11db69638945b087b842059d7e47c0, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:11.901744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342791b1-3198-4c08-bd1d-8116bceadde4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:11.921064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dfd28ddf-7864-456c-99b2-b584db50ca32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d118a5d4e6487f90744ecf58271823'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:11.928853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dfd28ddf-7864-456c-99b2-b584db50ca32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d118a5d4e6487f90744ecf58271823'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:11.945743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dfd28ddf-7864-456c-99b2-b584db50ca32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d118a5d4e6487f90744ecf58271823'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:12.021228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dfd28ddf-7864-456c-99b2-b584db50ca32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d118a5d4e6487f90744ecf58271823'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:12.022045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-dfd28ddf-7864-456c-99b2-b584db50ca32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d118a5d4e6487f90744ecf58271823'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:12.028837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1041/2079] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:11 2022] PUT /identity/v3/projects/d62c01d6a8ad4051b60c4a95a1ce13ae/users/bd11db69638945b087b842059d7e47c0/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:12.038125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1039/2080] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:11 2022] DELETE /identity/v3/users/e1d118a5d4e6487f90744ecf58271823 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:12.038699 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26303ca7-f011-47cf-8fd3-ffa09fea99ea None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:12.038699 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26303ca7-f011-47cf-8fd3-ffa09fea99ea None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:12.038699 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26303ca7-f011-47cf-8fd3-ffa09fea99ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:12.039638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8bb1-6baa-43aa-ba88-5b0870002ece None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:12.039934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8bb1-6baa-43aa-ba88-5b0870002ece None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:12.040215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8bb1-6baa-43aa-ba88-5b0870002ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:12.070282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-26303ca7-f011-47cf-8fd3-ffa09fea99ea None None] MFA Rules not processed for user `9431cd1afee04d408bceeb353b263241`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:12.071541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-26303ca7-f011-47cf-8fd3-ffa09fea99ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:12.073614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-f11d8bb1-6baa-43aa-ba88-5b0870002ece None None] MFA Rules not processed for user `bd11db69638945b087b842059d7e47c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:12.074854 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f11d8bb1-6baa-43aa-ba88-5b0870002ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:12.184986 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1040/2081] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:12.189386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d8bb1-6baa-43aa-ba88-5b0870002ece None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:12.195455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f11d8bb1-6baa-43aa-ba88-5b0870002ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:12.196924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1042/2082] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:12.201315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26303ca7-f011-47cf-8fd3-ffa09fea99ea None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:12.203660 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-26303ca7-f011-47cf-8fd3-ffa09fea99ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:12.302892 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26303ca7-f011-47cf-8fd3-ffa09fea99ea None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:12.312149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bafa5-dc00-4fe4-bfa0-e68b7411e99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:12.313654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bafa5-dc00-4fe4-bfa0-e68b7411e99d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:12.314014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bafa5-dc00-4fe4-bfa0-e68b7411e99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:12.314411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bafa5-dc00-4fe4-bfa0-e68b7411e99d admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:12.315206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bafa5-dc00-4fe4-bfa0-e68b7411e99d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:12.318568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bafa5-dc00-4fe4-bfa0-e68b7411e99d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:12.328909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d8bb1-6baa-43aa-ba88-5b0870002ece None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:12.353337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:12.362109 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2bdd5-fde5-4890-b51a-3d4a7813c4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:12.366113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2bdd5-fde5-4890-b51a-3d4a7813c4de admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:12.372588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2bdd5-fde5-4890-b51a-3d4a7813c4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:12.377645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2bdd5-fde5-4890-b51a-3d4a7813c4de admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:12.377645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2bdd5-fde5-4890-b51a-3d4a7813c4de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:12.398645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2bdd5-fde5-4890-b51a-3d4a7813c4de admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:12.463210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:12.463446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1043/2083] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:12 2022] POST /identity/v3/projects => generated 385 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:12.465228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bafa5-dc00-4fe4-bfa0-e68b7411e99d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:12.560346 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1041/2084] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:12 2022] POST /identity/v3/projects => generated 407 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:12.562930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2bdd5-fde5-4890-b51a-3d4a7813c4de admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:12.697981 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bafa5-dc00-4fe4-bfa0-e68b7411e99d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:12.744341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2bdd5-fde5-4890-b51a-3d4a7813c4de admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:12.775509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfca7b-6101-4771-a679-e87a09fed138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:12.777868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfca7b-6101-4771-a679-e87a09fed138 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:12.778169 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfca7b-6101-4771-a679-e87a09fed138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:12.778513 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfca7b-6101-4771-a679-e87a09fed138 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:12.779288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfca7b-6101-4771-a679-e87a09fed138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:12.782562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfca7b-6101-4771-a679-e87a09fed138 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:12.796785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589ed0f-f501-4135-b343-da2272decbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:12.803314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589ed0f-f501-4135-b343-da2272decbc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:12.803314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589ed0f-f501-4135-b343-da2272decbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:12.803314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589ed0f-f501-4135-b343-da2272decbc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:12.803314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589ed0f-f501-4135-b343-da2272decbc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:12.815551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8589ed0f-f501-4135-b343-da2272decbc7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:12.955305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1042/2085] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:12 2022] POST /identity/v3/users => generated 310 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:12.959260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfca7b-6101-4771-a679-e87a09fed138 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:12.960978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a6dfca7b-6101-4771-a679-e87a09fed138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:12.974567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfca7b-6101-4771-a679-e87a09fed138 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:12.979406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:12.981493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:12.981493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:12.985378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] PATH_INFO: `/v3/projects/4c86610a0dd14c2c986be65ad74ea79f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:12.985378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c86610a0dd14c2c986be65ad74ea79f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:12.986349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:12.992781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1044/2086] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:12 2022] POST /identity/v3/users => generated 299 bytes in 524 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:12.992781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589ed0f-f501-4135-b343-da2272decbc7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.011660 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8589ed0f-f501-4135-b343-da2272decbc7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.016900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad8fe6-c43c-4969-91e0-9943e4badea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.017596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad8fe6-c43c-4969-91e0-9943e4badea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.017872 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad8fe6-c43c-4969-91e0-9943e4badea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.018247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad8fe6-c43c-4969-91e0-9943e4badea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.019129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad8fe6-c43c-4969-91e0-9943e4badea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.020781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad8fe6-c43c-4969-91e0-9943e4badea0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.033494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1045/2087] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:13.033494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad8fe6-c43c-4969-91e0-9943e4badea0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.043455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad8fe6-c43c-4969-91e0-9943e4badea0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.048174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85487f65-7b2a-4da7-a47e-0b1325baafec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.048835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85487f65-7b2a-4da7-a47e-0b1325baafec admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.049254 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85487f65-7b2a-4da7-a47e-0b1325baafec admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.049534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85487f65-7b2a-4da7-a47e-0b1325baafec admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.052321 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85487f65-7b2a-4da7-a47e-0b1325baafec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.054007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85487f65-7b2a-4da7-a47e-0b1325baafec admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.065519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1046/2088] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:13.070298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85487f65-7b2a-4da7-a47e-0b1325baafec admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.079780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85487f65-7b2a-4da7-a47e-0b1325baafec admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.085418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a235d-6685-4c4c-b94f-724eb919435a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.086376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a235d-6685-4c4c-b94f-724eb919435a admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.086657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a235d-6685-4c4c-b94f-724eb919435a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.086932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a235d-6685-4c4c-b94f-724eb919435a admin admin] PATH_INFO: `/v3/projects/cd9e1e82103248398547480c762d59a9/users/08fb4c16a8b44c299939141c3308a7c3/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.107572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a235d-6685-4c4c-b94f-724eb919435a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd9e1e82103248398547480c762d59a9, user_id=08fb4c16a8b44c299939141c3308a7c3, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.110434 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a235d-6685-4c4c-b94f-724eb919435a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.215267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1047/2089] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:13 2022] PUT /identity/v3/projects/cd9e1e82103248398547480c762d59a9/users/08fb4c16a8b44c299939141c3308a7c3/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:13.215267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a235d-6685-4c4c-b94f-724eb919435a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.222931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c86610a0dd14c2c986be65ad74ea79f', 'request_id': 'req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:13.233023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c86610a0dd14c2c986be65ad74ea79f', 'request_id': 'req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:13.240856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c86610a0dd14c2c986be65ad74ea79f', 'request_id': 'req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:13.287166 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a235d-6685-4c4c-b94f-724eb919435a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.351627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81bde3-bfa9-4600-a5dd-33b2079cce7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.353015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81bde3-bfa9-4600-a5dd-33b2079cce7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.353015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81bde3-bfa9-4600-a5dd-33b2079cce7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.353204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] The token cache is being invalidate because project 4c86610a0dd14c2c986be65ad74ea79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:13.353628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81bde3-bfa9-4600-a5dd-33b2079cce7e admin admin] PATH_INFO: `/v3/projects/d0ad9f0355b24cfaaf1ea42eaae04832/users/7c43890dd4554c9fa80346ffaf3f910c/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.353733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:13.354266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:13.355877 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1043/2090] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:12 2022] DELETE /identity/v3/projects/4c86610a0dd14c2c986be65ad74ea79f => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:13.356661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.358305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:13.374975 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81bde3-bfa9-4600-a5dd-33b2079cce7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ad9f0355b24cfaaf1ea42eaae04832, user_id=7c43890dd4554c9fa80346ffaf3f910c, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.374975 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81bde3-bfa9-4600-a5dd-33b2079cce7e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.383338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf03651-b2eb-44b2-ab04-23e19a3f0e90 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.389258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fdd96-097b-4f81-8506-9cbeb11c3d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.390416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fdd96-097b-4f81-8506-9cbeb11c3d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.390416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fdd96-097b-4f81-8506-9cbeb11c3d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.390416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fdd96-097b-4f81-8506-9cbeb11c3d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.391110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fdd96-097b-4f81-8506-9cbeb11c3d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.392798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515fdd96-097b-4f81-8506-9cbeb11c3d99 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.413276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1044/2091] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:13.413276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87858a1-f041-46e1-bfa5-3bf4a1f558e5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.413276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87858a1-f041-46e1-bfa5-3bf4a1f558e5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.413276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87858a1-f041-46e1-bfa5-3bf4a1f558e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.430082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a87858a1-f041-46e1-bfa5-3bf4a1f558e5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:13.431266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a87858a1-f041-46e1-bfa5-3bf4a1f558e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:13.441109 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1045/2092] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:13.441798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87858a1-f041-46e1-bfa5-3bf4a1f558e5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.453083 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87858a1-f041-46e1-bfa5-3bf4a1f558e5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.463185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d767d-a9a7-4053-9d15-6f02a4b758c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.463185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d767d-a9a7-4053-9d15-6f02a4b758c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.463185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d767d-a9a7-4053-9d15-6f02a4b758c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.463185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d767d-a9a7-4053-9d15-6f02a4b758c2 admin admin] PATH_INFO: `/v3/projects/cd9e1e82103248398547480c762d59a9/users/08fb4c16a8b44c299939141c3308a7c3/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.463575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d767d-a9a7-4053-9d15-6f02a4b758c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd9e1e82103248398547480c762d59a9, user_id=08fb4c16a8b44c299939141c3308a7c3, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.464987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d767d-a9a7-4053-9d15-6f02a4b758c2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.478934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1046/2093] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:13 2022] PUT /identity/v3/projects/cd9e1e82103248398547480c762d59a9/users/08fb4c16a8b44c299939141c3308a7c3/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:13.480273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61416553-9665-4e6c-930a-ebaa03262e1f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.480571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61416553-9665-4e6c-930a-ebaa03262e1f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.480856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61416553-9665-4e6c-930a-ebaa03262e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.502612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1048/2094] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:13 2022] PUT /identity/v3/projects/d0ad9f0355b24cfaaf1ea42eaae04832/users/7c43890dd4554c9fa80346ffaf3f910c/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:13.504201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c0425-6113-4fbf-983c-dcebb92337fc None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.504554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c0425-6113-4fbf-983c-dcebb92337fc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.504840 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c0425-6113-4fbf-983c-dcebb92337fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.518025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-61416553-9665-4e6c-930a-ebaa03262e1f None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:13.525069 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-61416553-9665-4e6c-930a-ebaa03262e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:13.536335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ef6c0425-6113-4fbf-983c-dcebb92337fc None None] MFA Rules not processed for user `08fb4c16a8b44c299939141c3308a7c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:13.537691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ef6c0425-6113-4fbf-983c-dcebb92337fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:13.629130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1049/2095] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:13 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:13.630642 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623e38-a267-4670-91a3-77194817a5e2 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.631011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623e38-a267-4670-91a3-77194817a5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.631523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623e38-a267-4670-91a3-77194817a5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.659184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1047/2096] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:13.660056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61416553-9665-4e6c-930a-ebaa03262e1f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.662869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-61416553-9665-4e6c-930a-ebaa03262e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:13.671733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a7623e38-a267-4670-91a3-77194817a5e2 None None] MFA Rules not processed for user `8617eb9fff41436d867e205da136d61f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:13.673581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a7623e38-a267-4670-91a3-77194817a5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:13.677163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61416553-9665-4e6c-930a-ebaa03262e1f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.684257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d6cb1-c815-4aa1-af95-20691dcc4ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.685276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6cb1-c815-4aa1-af95-20691dcc4ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.685566 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6cb1-c815-4aa1-af95-20691dcc4ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.685845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6cb1-c815-4aa1-af95-20691dcc4ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.686605 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d6cb1-c815-4aa1-af95-20691dcc4ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.688448 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d6cb1-c815-4aa1-af95-20691dcc4ea6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.702453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1048/2097] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:13.702979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d6cb1-c815-4aa1-af95-20691dcc4ea6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.704699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ef1d6cb1-c815-4aa1-af95-20691dcc4ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:13.732101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:13.753702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1050/2098] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:13.755518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:13.758998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623e38-a267-4670-91a3-77194817a5e2 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.773009 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d6cb1-c815-4aa1-af95-20691dcc4ea6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.773787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623e38-a267-4670-91a3-77194817a5e2 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.778459 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d95a2-14f4-436c-84ed-f3eea5c75dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.779149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d95a2-14f4-436c-84ed-f3eea5c75dba admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.779437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d95a2-14f4-436c-84ed-f3eea5c75dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.779709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d95a2-14f4-436c-84ed-f3eea5c75dba admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.780243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e405b-2064-4a5e-8cf1-aa24c876a45a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.780450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d95a2-14f4-436c-84ed-f3eea5c75dba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:13.780752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d95a2-14f4-436c-84ed-f3eea5c75dba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.780849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e405b-2064-4a5e-8cf1-aa24c876a45a service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.781158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e405b-2064-4a5e-8cf1-aa24c876a45a service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.781442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e405b-2064-4a5e-8cf1-aa24c876a45a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.782482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d95a2-14f4-436c-84ed-f3eea5c75dba admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.792235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e405b-2064-4a5e-8cf1-aa24c876a45a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.793746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e405b-2064-4a5e-8cf1-aa24c876a45a service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.796614 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1051/2099] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:13.797464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d95a2-14f4-436c-84ed-f3eea5c75dba admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.807723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1049/2100] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:13 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:13.808566 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e405b-2064-4a5e-8cf1-aa24c876a45a service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.813024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d95a2-14f4-436c-84ed-f3eea5c75dba admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.814513 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d04e405b-2064-4a5e-8cf1-aa24c876a45a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:13.831147 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d01fb-47e9-44b0-96cb-1b3d2930c757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.831147 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d01fb-47e9-44b0-96cb-1b3d2930c757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.831147 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d01fb-47e9-44b0-96cb-1b3d2930c757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.831147 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d01fb-47e9-44b0-96cb-1b3d2930c757 admin admin] PATH_INFO: `/v3/projects/d0ad9f0355b24cfaaf1ea42eaae04832/users/7c43890dd4554c9fa80346ffaf3f910c/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.833143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d01fb-47e9-44b0-96cb-1b3d2930c757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ad9f0355b24cfaaf1ea42eaae04832, user_id=7c43890dd4554c9fa80346ffaf3f910c, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.833143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d01fb-47e9-44b0-96cb-1b3d2930c757 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.833278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e405b-2064-4a5e-8cf1-aa24c876a45a service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.841102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1052/2101] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:13 2022] PUT /identity/v3/projects/d0ad9f0355b24cfaaf1ea42eaae04832/users/7c43890dd4554c9fa80346ffaf3f910c/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:13.842000 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d01fb-47e9-44b0-96cb-1b3d2930c757 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:13.926245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574966d-cdb4-4502-9ba4-3f9f80dfc302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.927613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574966d-cdb4-4502-9ba4-3f9f80dfc302 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.928463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574966d-cdb4-4502-9ba4-3f9f80dfc302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.930010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574966d-cdb4-4502-9ba4-3f9f80dfc302 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.931400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574966d-cdb4-4502-9ba4-3f9f80dfc302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.939802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574966d-cdb4-4502-9ba4-3f9f80dfc302 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:13.947028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d01fb-47e9-44b0-96cb-1b3d2930c757 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:13.953069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a252b-e84d-43c4-bd79-0cd72cae8296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:13.955472 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a252b-e84d-43c4-bd79-0cd72cae8296 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:13.955947 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a252b-e84d-43c4-bd79-0cd72cae8296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:13.956401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a252b-e84d-43c4-bd79-0cd72cae8296 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:13.957988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a252b-e84d-43c4-bd79-0cd72cae8296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:13.959969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08a252b-e84d-43c4-bd79-0cd72cae8296 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.052128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1053/2102] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:13 2022] POST /identity/v3/projects => generated 399 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:14.056389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1050/2103] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:13 2022] POST /identity/v3/projects => generated 373 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:14.056577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32e3b7-c038-495f-938e-ec1d76110e62 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.056577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32e3b7-c038-495f-938e-ec1d76110e62 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.056577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32e3b7-c038-495f-938e-ec1d76110e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.056820 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574966d-cdb4-4502-9ba4-3f9f80dfc302 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.083202 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0f32e3b7-c038-495f-938e-ec1d76110e62 None None] MFA Rules not processed for user `7c43890dd4554c9fa80346ffaf3f910c`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:14.089082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0f32e3b7-c038-495f-938e-ec1d76110e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:14.121956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574966d-cdb4-4502-9ba4-3f9f80dfc302 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.126793 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49203c3-e4d9-4403-8dbb-437939d9b078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.127550 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49203c3-e4d9-4403-8dbb-437939d9b078 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.127771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49203c3-e4d9-4403-8dbb-437939d9b078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.128052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49203c3-e4d9-4403-8dbb-437939d9b078 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.128789 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49203c3-e4d9-4403-8dbb-437939d9b078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.131319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49203c3-e4d9-4403-8dbb-437939d9b078 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.137690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:14.165063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1054/2104] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:14.166024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32e3b7-c038-495f-938e-ec1d76110e62 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.175935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32e3b7-c038-495f-938e-ec1d76110e62 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.180817 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49d27c-f19f-4ef6-9fe7-6fe390f02a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.181429 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49d27c-f19f-4ef6-9fe7-6fe390f02a7c admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.181717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49d27c-f19f-4ef6-9fe7-6fe390f02a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.181994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49d27c-f19f-4ef6-9fe7-6fe390f02a7c admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.182688 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49d27c-f19f-4ef6-9fe7-6fe390f02a7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.184724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49d27c-f19f-4ef6-9fe7-6fe390f02a7c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.194565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:14.225279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1051/2105] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:14 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:14.225279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49203c3-e4d9-4403-8dbb-437939d9b078 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.225279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d49203c3-e4d9-4403-8dbb-437939d9b078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:14.285347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49203c3-e4d9-4403-8dbb-437939d9b078 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.289186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c596757-fcf7-4002-8181-a5f301e8cc6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.289873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c596757-fcf7-4002-8181-a5f301e8cc6e service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.290153 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c596757-fcf7-4002-8181-a5f301e8cc6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.290850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c596757-fcf7-4002-8181-a5f301e8cc6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.292424 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7c596757-fcf7-4002-8181-a5f301e8cc6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:14.302890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1055/2106] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:14 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:14.303807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49d27c-f19f-4ef6-9fe7-6fe390f02a7c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.317839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49d27c-f19f-4ef6-9fe7-6fe390f02a7c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.322664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3afe2-666d-4843-ae92-3a9ac1612b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.323354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3afe2-666d-4843-ae92-3a9ac1612b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.323704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3afe2-666d-4843-ae92-3a9ac1612b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.324013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3afe2-666d-4843-ae92-3a9ac1612b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.324757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3afe2-666d-4843-ae92-3a9ac1612b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.326487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3afe2-666d-4843-ae92-3a9ac1612b71 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.338912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1056/2107] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:14.338912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3afe2-666d-4843-ae92-3a9ac1612b71 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.351743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3afe2-666d-4843-ae92-3a9ac1612b71 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.356921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f9fa4-0a9d-4230-ab45-5cffeece0a4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.357324 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c596757-fcf7-4002-8181-a5f301e8cc6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.357777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f9fa4-0a9d-4230-ab45-5cffeece0a4f service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.357777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f9fa4-0a9d-4230-ab45-5cffeece0a4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.358051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f9fa4-0a9d-4230-ab45-5cffeece0a4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.358891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c596757-fcf7-4002-8181-a5f301e8cc6e service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.360059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-280f9fa4-0a9d-4230-ab45-5cffeece0a4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:14.369251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1052/2108] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:14.374459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c596757-fcf7-4002-8181-a5f301e8cc6e service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.382105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c596757-fcf7-4002-8181-a5f301e8cc6e service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.388339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd785c2-c183-4977-894d-f6f9c56ff4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.389140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd785c2-c183-4977-894d-f6f9c56ff4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.389529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd785c2-c183-4977-894d-f6f9c56ff4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.389883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd785c2-c183-4977-894d-f6f9c56ff4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.390940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd785c2-c183-4977-894d-f6f9c56ff4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.393632 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd785c2-c183-4977-894d-f6f9c56ff4c1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.414950 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1053/2109] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:14.416099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd785c2-c183-4977-894d-f6f9c56ff4c1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.429095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd785c2-c183-4977-894d-f6f9c56ff4c1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.435401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0679d-a7cf-46e3-88b0-87f7c15121b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.436038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0679d-a7cf-46e3-88b0-87f7c15121b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.436327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0679d-a7cf-46e3-88b0-87f7c15121b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.436647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0679d-a7cf-46e3-88b0-87f7c15121b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.437550 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0679d-a7cf-46e3-88b0-87f7c15121b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.439199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0679d-a7cf-46e3-88b0-87f7c15121b6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.451564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1054/2110] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:14.455096 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0679d-a7cf-46e3-88b0-87f7c15121b6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.455198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f9fa4-0a9d-4230-ab45-5cffeece0a4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.455198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f9fa4-0a9d-4230-ab45-5cffeece0a4f service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.468235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1057/2111] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:14 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:14.469355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f9fa4-0a9d-4230-ab45-5cffeece0a4f service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.475029 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0679d-a7cf-46e3-88b0-87f7c15121b6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.481866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f9fa4-0a9d-4230-ab45-5cffeece0a4f service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.485579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c3349-4aaa-4326-b7db-e0ab3f7c91f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.486785 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c3349-4aaa-4326-b7db-e0ab3f7c91f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.487046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fdcd9-ca9d-47f2-85af-a891687c8b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.487178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c3349-4aaa-4326-b7db-e0ab3f7c91f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.487394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c3349-4aaa-4326-b7db-e0ab3f7c91f3 admin admin] PATH_INFO: `/v3/projects/a6cce398ee6f4f5183e7f24c3da5f70b/users/d572114f5f234aa0a399494644d47254/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.488278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fdcd9-ca9d-47f2-85af-a891687c8b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.488528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fdcd9-ca9d-47f2-85af-a891687c8b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.489721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fdcd9-ca9d-47f2-85af-a891687c8b4b admin admin] PATH_INFO: `/v3/projects/ad2e38932a1447ad90cdefcd0004afa7/users/72fc5d813ddf48108917fd91130a72d5/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.508985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fdcd9-ca9d-47f2-85af-a891687c8b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad2e38932a1447ad90cdefcd0004afa7, user_id=72fc5d813ddf48108917fd91130a72d5, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.512512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fdcd9-ca9d-47f2-85af-a891687c8b4b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.525155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c3349-4aaa-4326-b7db-e0ab3f7c91f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6cce398ee6f4f5183e7f24c3da5f70b, user_id=d572114f5f234aa0a399494644d47254, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.531025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c3349-4aaa-4326-b7db-e0ab3f7c91f3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.612589 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1055/2112] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:14 2022] PUT /identity/v3/projects/a6cce398ee6f4f5183e7f24c3da5f70b/users/d572114f5f234aa0a399494644d47254/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:14.618262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c3349-4aaa-4326-b7db-e0ab3f7c91f3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.623256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1058/2113] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:14 2022] PUT /identity/v3/projects/ad2e38932a1447ad90cdefcd0004afa7/users/72fc5d813ddf48108917fd91130a72d5/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:14.634152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9a4be-21a8-43b2-93e7-6869b2b2c277 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.634648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9a4be-21a8-43b2-93e7-6869b2b2c277 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.635368 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9a4be-21a8-43b2-93e7-6869b2b2c277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.702573 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c0a9a4be-21a8-43b2-93e7-6869b2b2c277 None None] MFA Rules not processed for user `72fc5d813ddf48108917fd91130a72d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:14.704192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c0a9a4be-21a8-43b2-93e7-6869b2b2c277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:14.717485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c3349-4aaa-4326-b7db-e0ab3f7c91f3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.722536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658dac1-bf38-490e-9bef-fc35857fa3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.724197 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658dac1-bf38-490e-9bef-fc35857fa3ae admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.724484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658dac1-bf38-490e-9bef-fc35857fa3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.724933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658dac1-bf38-490e-9bef-fc35857fa3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.725826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658dac1-bf38-490e-9bef-fc35857fa3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.727337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658dac1-bf38-490e-9bef-fc35857fa3ae admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.737384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:14.747906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1056/2114] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:14.752385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658dac1-bf38-490e-9bef-fc35857fa3ae admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.764077 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658dac1-bf38-490e-9bef-fc35857fa3ae admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.769476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bb334-3444-402c-97a5-03243e48fa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.770424 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bb334-3444-402c-97a5-03243e48fa7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.770714 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bb334-3444-402c-97a5-03243e48fa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.771004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bb334-3444-402c-97a5-03243e48fa7f admin admin] PATH_INFO: `/v3/projects/a6cce398ee6f4f5183e7f24c3da5f70b/users/d572114f5f234aa0a399494644d47254/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.773247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bb334-3444-402c-97a5-03243e48fa7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6cce398ee6f4f5183e7f24c3da5f70b, user_id=d572114f5f234aa0a399494644d47254, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.777330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bb334-3444-402c-97a5-03243e48fa7f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.788985 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1057/2115] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:14 2022] PUT /identity/v3/projects/a6cce398ee6f4f5183e7f24c3da5f70b/users/d572114f5f234aa0a399494644d47254/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:14.797104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1059/2116] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:14.804332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bb334-3444-402c-97a5-03243e48fa7f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.876663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bb334-3444-402c-97a5-03243e48fa7f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:14.881535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688333d-28f7-4f5b-913b-83d33726ba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:14.882219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688333d-28f7-4f5b-913b-83d33726ba4e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:14.882497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688333d-28f7-4f5b-913b-83d33726ba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:14.882771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688333d-28f7-4f5b-913b-83d33726ba4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:14.883473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e688333d-28f7-4f5b-913b-83d33726ba4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:14.885436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e688333d-28f7-4f5b-913b-83d33726ba4e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:14.986791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1058/2117] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:14 2022] POST /identity/v3/projects => generated 399 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:14.987422 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9a4be-21a8-43b2-93e7-6869b2b2c277 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:14.998773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9a4be-21a8-43b2-93e7-6869b2b2c277 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:15.085463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2ff25-a22a-4115-8b22-216ef1712a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:15.088026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2ff25-a22a-4115-8b22-216ef1712a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:15.088849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2ff25-a22a-4115-8b22-216ef1712a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:15.089298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2ff25-a22a-4115-8b22-216ef1712a3b admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:15.090493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2ff25-a22a-4115-8b22-216ef1712a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:15.093378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2ff25-a22a-4115-8b22-216ef1712a3b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:15.101636 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:15.219864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1060/2118] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:14 2022] POST /identity/v3/users => generated 306 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:15.221760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688333d-28f7-4f5b-913b-83d33726ba4e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:15.434564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688333d-28f7-4f5b-913b-83d33726ba4e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:15.434564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e72943-c2e0-462e-82d0-1c47958deb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:15.434564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e72943-c2e0-462e-82d0-1c47958deb00 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:15.434564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e72943-c2e0-462e-82d0-1c47958deb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:15.438719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e72943-c2e0-462e-82d0-1c47958deb00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:15.438719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e72943-c2e0-462e-82d0-1c47958deb00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:15.438719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e72943-c2e0-462e-82d0-1c47958deb00 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:15.438719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1059/2119] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:15.440151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2ff25-a22a-4115-8b22-216ef1712a3b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:15.440151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2ff25-a22a-4115-8b22-216ef1712a3b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:15.440151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0a549-8bf4-4e5d-b749-03745e570644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:15.440151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0a549-8bf4-4e5d-b749-03745e570644 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:15.441269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0a549-8bf4-4e5d-b749-03745e570644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:15.441269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0a549-8bf4-4e5d-b749-03745e570644 admin admin] PATH_INFO: `/v3/projects/00f85c7409ed43b7ad1badbdc4e05992/users/42fb5659541742ecb711f3e320642993/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:15.441269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0a549-8bf4-4e5d-b749-03745e570644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f85c7409ed43b7ad1badbdc4e05992, user_id=42fb5659541742ecb711f3e320642993, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:15.441269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0a549-8bf4-4e5d-b749-03745e570644 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:16.066285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1061/2120] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:15 2022] PUT /identity/v3/projects/00f85c7409ed43b7ad1badbdc4e05992/users/42fb5659541742ecb711f3e320642993/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:16.068606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e72943-c2e0-462e-82d0-1c47958deb00 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:16.079089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e72943-c2e0-462e-82d0-1c47958deb00 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:16.154396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7d95-c25c-47fd-8eac-db40edf9a120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:16.154396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7d95-c25c-47fd-8eac-db40edf9a120 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:16.165300 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7d95-c25c-47fd-8eac-db40edf9a120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:16.165872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7d95-c25c-47fd-8eac-db40edf9a120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:16.171321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed7d95-c25c-47fd-8eac-db40edf9a120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:16.177445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed7d95-c25c-47fd-8eac-db40edf9a120 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:16.224333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1060/2121] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:16.225325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0a549-8bf4-4e5d-b749-03745e570644 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:16.251375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0a549-8bf4-4e5d-b749-03745e570644 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:16.268924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac81e8-1cbf-4ae9-bd4b-c6cc54b29510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:16.279256 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac81e8-1cbf-4ae9-bd4b-c6cc54b29510 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:16.282013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac81e8-1cbf-4ae9-bd4b-c6cc54b29510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:16.286891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac81e8-1cbf-4ae9-bd4b-c6cc54b29510 admin admin] PATH_INFO: `/v3/projects/00f85c7409ed43b7ad1badbdc4e05992/users/42fb5659541742ecb711f3e320642993/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:16.288949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac81e8-1cbf-4ae9-bd4b-c6cc54b29510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f85c7409ed43b7ad1badbdc4e05992, user_id=42fb5659541742ecb711f3e320642993, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:16.292917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac81e8-1cbf-4ae9-bd4b-c6cc54b29510 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:16.316440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1062/2122] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:16 2022] PUT /identity/v3/projects/00f85c7409ed43b7ad1badbdc4e05992/users/42fb5659541742ecb711f3e320642993/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:16.329001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7c43c-0732-4de8-9dfe-6f78bad5a044 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:16.329326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7c43c-0732-4de8-9dfe-6f78bad5a044 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:16.329576 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7c43c-0732-4de8-9dfe-6f78bad5a044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:16.368820 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d9d7c43c-0732-4de8-9dfe-6f78bad5a044 None None] MFA Rules not processed for user `42fb5659541742ecb711f3e320642993`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:16.374186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d9d7c43c-0732-4de8-9dfe-6f78bad5a044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:16.485373 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1061/2123] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:16 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:16.495481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac81e8-1cbf-4ae9-bd4b-c6cc54b29510 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:16.498629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:16.563841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac81e8-1cbf-4ae9-bd4b-c6cc54b29510 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:16.567998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1fc8f-1de1-43fd-893c-631adb9bb7dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:16.568517 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1fc8f-1de1-43fd-893c-631adb9bb7dc service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:16.568803 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1fc8f-1de1-43fd-893c-631adb9bb7dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:16.569165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1fc8f-1de1-43fd-893c-631adb9bb7dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:16.584220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1fc8f-1de1-43fd-893c-631adb9bb7dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:16.587469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1fc8f-1de1-43fd-893c-631adb9bb7dc service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:16.597162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1063/2124] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:16 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:16.779896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7c43c-0732-4de8-9dfe-6f78bad5a044 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:16.789471 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7c43c-0732-4de8-9dfe-6f78bad5a044 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:16.793730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5086a6-4f64-44d3-aa87-78eebe0e5344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:16.793881 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5086a6-4f64-44d3-aa87-78eebe0e5344 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:16.794129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5086a6-4f64-44d3-aa87-78eebe0e5344 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:16.794405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5086a6-4f64-44d3-aa87-78eebe0e5344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:16.860677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5086a6-4f64-44d3-aa87-78eebe0e5344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:16.862214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5086a6-4f64-44d3-aa87-78eebe0e5344 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:16.873277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1062/2125] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:17.161204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a112d36-176c-4c9b-9ec0-21abd405d1fb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:17.161851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a112d36-176c-4c9b-9ec0-21abd405d1fb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:17.162113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a112d36-176c-4c9b-9ec0-21abd405d1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:17.193616 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8a112d36-176c-4c9b-9ec0-21abd405d1fb None None] MFA Rules not processed for user `d572114f5f234aa0a399494644d47254`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:17.195013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8a112d36-176c-4c9b-9ec0-21abd405d1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:17.290880 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1064/2126] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:17 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:17.656205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5086a6-4f64-44d3-aa87-78eebe0e5344 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:17.666772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5086a6-4f64-44d3-aa87-78eebe0e5344 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:17.670895 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5314a0-452e-437f-8a5c-0704bc1c4f28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:17.671377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5314a0-452e-437f-8a5c-0704bc1c4f28 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:17.671663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5314a0-452e-437f-8a5c-0704bc1c4f28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:17.671941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5314a0-452e-437f-8a5c-0704bc1c4f28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:17.731005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5314a0-452e-437f-8a5c-0704bc1c4f28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:17.732710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5314a0-452e-437f-8a5c-0704bc1c4f28 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:17.742660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1063/2127] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:17.858085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:17.960093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:19.318362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:20.086454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:23.790423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a112d36-176c-4c9b-9ec0-21abd405d1fb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:23.798964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a112d36-176c-4c9b-9ec0-21abd405d1fb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:23.804524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb312154-50da-460b-9e4f-3ad74a3367e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:23.805320 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb312154-50da-460b-9e4f-3ad74a3367e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:23.805768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb312154-50da-460b-9e4f-3ad74a3367e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:23.806061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb312154-50da-460b-9e4f-3ad74a3367e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:23.816086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb312154-50da-460b-9e4f-3ad74a3367e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:23.818891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb312154-50da-460b-9e4f-3ad74a3367e1 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:23.832849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1065/2128] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:26.064432 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5314a0-452e-437f-8a5c-0704bc1c4f28 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:26.066058 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb312154-50da-460b-9e4f-3ad74a3367e1 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:26.074994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5314a0-452e-437f-8a5c-0704bc1c4f28 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:26.081192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf303ab3-4792-488d-a458-a71bc948bcd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:26.082440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf303ab3-4792-488d-a458-a71bc948bcd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:26.082754 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf303ab3-4792-488d-a458-a71bc948bcd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:26.083013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf303ab3-4792-488d-a458-a71bc948bcd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:26.084830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:26.128555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:26.146428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb312154-50da-460b-9e4f-3ad74a3367e1 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:26.151545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:26.151784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf303ab3-4792-488d-a458-a71bc948bcd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:26.152423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:26.152696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:26.153182 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] PATH_INFO: `/v3/users/72fc5d813ddf48108917fd91130a72d5` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:26.153356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf303ab3-4792-488d-a458-a71bc948bcd5 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:26.154563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72fc5d813ddf48108917fd91130a72d5)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:26.156519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:26.169094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1064/2129] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:26.333491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72fc5d813ddf48108917fd91130a72d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.346208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72fc5d813ddf48108917fd91130a72d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.353830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72fc5d813ddf48108917fd91130a72d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.471673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72fc5d813ddf48108917fd91130a72d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.472309 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72fc5d813ddf48108917fd91130a72d5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.480065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1066/2130] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:26 2022] DELETE /identity/v3/users/72fc5d813ddf48108917fd91130a72d5 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:26.485709 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd9405-d4d2-4ffe-900f-8a99c4f39f2c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:26.486081 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd9405-d4d2-4ffe-900f-8a99c4f39f2c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:26.486392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd9405-d4d2-4ffe-900f-8a99c4f39f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:26.505108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2bcd9405-d4d2-4ffe-900f-8a99c4f39f2c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:26.506371 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2bcd9405-d4d2-4ffe-900f-8a99c4f39f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:26.570341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1065/2131] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:26.582306 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:26.583837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:26.650863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d158fc-ffc2-44b4-91fd-086669cc0f78 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:26.654631 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8edf8-d747-430b-a4db-787b195af472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:26.655284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8edf8-d747-430b-a4db-787b195af472 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:26.655567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8edf8-d747-430b-a4db-787b195af472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:26.655847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8edf8-d747-430b-a4db-787b195af472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:26.669232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8edf8-d747-430b-a4db-787b195af472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:26.669232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8edf8-d747-430b-a4db-787b195af472 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:26.678378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1067/2132] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:26.689363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd9405-d4d2-4ffe-900f-8a99c4f39f2c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:26.691607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2bcd9405-d4d2-4ffe-900f-8a99c4f39f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:26.710555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd9405-d4d2-4ffe-900f-8a99c4f39f2c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:26.719612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:26.719612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:26.719612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:26.719612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] PATH_INFO: `/v3/users/1d4ee9f78817409d9fc839ec3cc68f7b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:26.720044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d4ee9f78817409d9fc839ec3cc68f7b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:26.721276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:26.935354 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d4ee9f78817409d9fc839ec3cc68f7b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.947463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d4ee9f78817409d9fc839ec3cc68f7b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:26.963390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d4ee9f78817409d9fc839ec3cc68f7b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.026486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d4ee9f78817409d9fc839ec3cc68f7b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.027056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d4ee9f78817409d9fc839ec3cc68f7b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.034065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1066/2133] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:26 2022] DELETE /identity/v3/users/1d4ee9f78817409d9fc839ec3cc68f7b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:27.039561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fce019-fa3a-461a-8c32-5136b874d4a3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:27.039815 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fce019-fa3a-461a-8c32-5136b874d4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:27.040112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fce019-fa3a-461a-8c32-5136b874d4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:27.061225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a0fce019-fa3a-461a-8c32-5136b874d4a3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:27.062455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a0fce019-fa3a-461a-8c32-5136b874d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:27.098398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:27.133088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1068/2134] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:27.145356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:27.146818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:27.189484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fce019-fa3a-461a-8c32-5136b874d4a3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:27.190953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a0fce019-fa3a-461a-8c32-5136b874d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:27.211467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fce019-fa3a-461a-8c32-5136b874d4a3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:27.226348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ab8b1-bce4-4c6f-a14e-398cc96aa7ee admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:27.226971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:27.227853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:27.228146 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:27.228426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] PATH_INFO: `/v3/projects/ad2e38932a1447ad90cdefcd0004afa7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:27.229859 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad2e38932a1447ad90cdefcd0004afa7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:27.230858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa790508-3183-4794-84f3-14d8cc49a5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:27.231590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa790508-3183-4794-84f3-14d8cc49a5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:27.231783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:27.232023 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa790508-3183-4794-84f3-14d8cc49a5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:27.232446 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa790508-3183-4794-84f3-14d8cc49a5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:27.241372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:27.245883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa790508-3183-4794-84f3-14d8cc49a5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:27.248724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa790508-3183-4794-84f3-14d8cc49a5a3 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:27.257811 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1067/2135] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:27.354177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad2e38932a1447ad90cdefcd0004afa7', 'request_id': 'req-4ae0e261-0817-4f20-87a0-5e13ee3e638a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.431637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad2e38932a1447ad90cdefcd0004afa7', 'request_id': 'req-4ae0e261-0817-4f20-87a0-5e13ee3e638a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.438162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad2e38932a1447ad90cdefcd0004afa7', 'request_id': 'req-4ae0e261-0817-4f20-87a0-5e13ee3e638a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.450607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] The token cache is being invalidate because project ad2e38932a1447ad90cdefcd0004afa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:27.450895 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.451837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.453409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1069/2136] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:27 2022] DELETE /identity/v3/projects/ad2e38932a1447ad90cdefcd0004afa7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:27.459576 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa790508-3183-4794-84f3-14d8cc49a5a3 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:27.470056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa790508-3183-4794-84f3-14d8cc49a5a3 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:27.533284 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f0101-dae7-42fc-883c-9b80cd336a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:27.534289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0101-dae7-42fc-883c-9b80cd336a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:27.534627 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0101-dae7-42fc-883c-9b80cd336a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:27.535548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f0101-dae7-42fc-883c-9b80cd336a02 admin admin] PATH_INFO: `/v3/users/42fb5659541742ecb711f3e320642993` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:27.536784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f0101-dae7-42fc-883c-9b80cd336a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42fb5659541742ecb711f3e320642993)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:27.539059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f0101-dae7-42fc-883c-9b80cd336a02 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:27.668271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:27.669768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:27.684662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0e261-0817-4f20-87a0-5e13ee3e638a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:27.689441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:27.690338 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:27.690654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:27.690936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] PATH_INFO: `/v3/projects/1a9f6af39f8d490c9a232f82d46d4d08` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:27.692199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a9f6af39f8d490c9a232f82d46d4d08)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:27.694342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:27.721655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6c9f0101-dae7-42fc-883c-9b80cd336a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42fb5659541742ecb711f3e320642993'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.729914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6c9f0101-dae7-42fc-883c-9b80cd336a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42fb5659541742ecb711f3e320642993'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.742902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6c9f0101-dae7-42fc-883c-9b80cd336a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42fb5659541742ecb711f3e320642993'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.792742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a9f6af39f8d490c9a232f82d46d4d08', 'request_id': 'req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.804115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6c9f0101-dae7-42fc-883c-9b80cd336a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42fb5659541742ecb711f3e320642993'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.805249 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6c9f0101-dae7-42fc-883c-9b80cd336a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42fb5659541742ecb711f3e320642993'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.817581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1068/2137] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:27 2022] DELETE /identity/v3/users/42fb5659541742ecb711f3e320642993 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:27.872363 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a9f6af39f8d490c9a232f82d46d4d08', 'request_id': 'req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.880399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a9f6af39f8d490c9a232f82d46d4d08', 'request_id': 'req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.895553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] The token cache is being invalidate because project 1a9f6af39f8d490c9a232f82d46d4d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:27.895864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.897802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ef90d1a-8ba5-40ca-b0af-100dd8086c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:27.902260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1070/2138] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:27 2022] DELETE /identity/v3/projects/1a9f6af39f8d490c9a232f82d46d4d08 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:27.906014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5739972-1d42-4484-a054-b9a1fef89087 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:27.906766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5739972-1d42-4484-a054-b9a1fef89087 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:27.907147 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5739972-1d42-4484-a054-b9a1fef89087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:27.943610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c5739972-1d42-4484-a054-b9a1fef89087 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:27.944984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c5739972-1d42-4484-a054-b9a1fef89087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:28.026691 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1069/2139] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:28.034196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e528247-5835-4602-a953-9185436768e6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.034491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e528247-5835-4602-a953-9185436768e6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.034777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e528247-5835-4602-a953-9185436768e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.061265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6e528247-5835-4602-a953-9185436768e6 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:28.062478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6e528247-5835-4602-a953-9185436768e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:28.073213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1071/2140] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:28.077499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5739972-1d42-4484-a054-b9a1fef89087 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:28.089389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5739972-1d42-4484-a054-b9a1fef89087 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:28.094158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d5653-edc0-4118-8a37-bd5f20f679bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:28.096481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d5653-edc0-4118-8a37-bd5f20f679bd admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.096481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d5653-edc0-4118-8a37-bd5f20f679bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.096481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d5653-edc0-4118-8a37-bd5f20f679bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.096481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d5653-edc0-4118-8a37-bd5f20f679bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:28.096833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d5653-edc0-4118-8a37-bd5f20f679bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:28.098849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d5653-edc0-4118-8a37-bd5f20f679bd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:28.110161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:28.115301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1070/2141] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:28.119701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e528247-5835-4602-a953-9185436768e6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:28.131948 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e528247-5835-4602-a953-9185436768e6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:28.139298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b9a16-0142-4cb0-8dc6-b2b813ccd540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:28.140201 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b9a16-0142-4cb0-8dc6-b2b813ccd540 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.140277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b9a16-0142-4cb0-8dc6-b2b813ccd540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.140628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b9a16-0142-4cb0-8dc6-b2b813ccd540 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.141299 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b9a16-0142-4cb0-8dc6-b2b813ccd540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:28.143273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b9a16-0142-4cb0-8dc6-b2b813ccd540 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:28.266992 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1072/2142] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:28 2022] POST /identity/v3/projects => generated 371 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:28.271415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d5653-edc0-4118-8a37-bd5f20f679bd admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:28.285604 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d5653-edc0-4118-8a37-bd5f20f679bd admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:28.361277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b9a16-0142-4cb0-8dc6-b2b813ccd540 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:28.363711 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fd6b9a16-0142-4cb0-8dc6-b2b813ccd540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:28.380482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ccdaa-e561-418e-93e5-d390be7dac20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:28.381228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccdaa-e561-418e-93e5-d390be7dac20 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.381518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccdaa-e561-418e-93e5-d390be7dac20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.381959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccdaa-e561-418e-93e5-d390be7dac20 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.382676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ccdaa-e561-418e-93e5-d390be7dac20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:28.383739 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:28.386101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ccdaa-e561-418e-93e5-d390be7dac20 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:28.449257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b9a16-0142-4cb0-8dc6-b2b813ccd540 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:28.454435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:28.455298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.455576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.455848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] PATH_INFO: `/v3/projects/00f85c7409ed43b7ad1badbdc4e05992` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.457671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f85c7409ed43b7ad1badbdc4e05992)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:28.459625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:28.517940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1071/2143] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:28 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:28.522700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ccdaa-e561-418e-93e5-d390be7dac20 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:28.532843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ccdaa-e561-418e-93e5-d390be7dac20 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:28.538286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52acc63-71f0-458d-8b46-dba2cb377ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:28.538949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52acc63-71f0-458d-8b46-dba2cb377ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.539227 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52acc63-71f0-458d-8b46-dba2cb377ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.539506 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52acc63-71f0-458d-8b46-dba2cb377ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.540235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52acc63-71f0-458d-8b46-dba2cb377ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:28.541879 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52acc63-71f0-458d-8b46-dba2cb377ba8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:28.552062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1072/2144] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:28.556231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52acc63-71f0-458d-8b46-dba2cb377ba8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:28.566248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52acc63-71f0-458d-8b46-dba2cb377ba8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:28.571517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3b2fb-cd59-4ac8-90c2-7413c1a72354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:28.572186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3b2fb-cd59-4ac8-90c2-7413c1a72354 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.572471 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3b2fb-cd59-4ac8-90c2-7413c1a72354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.572762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3b2fb-cd59-4ac8-90c2-7413c1a72354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.582310 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3b2fb-cd59-4ac8-90c2-7413c1a72354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:28.582310 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3b2fb-cd59-4ac8-90c2-7413c1a72354 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:28.587508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1073/2145] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:28.587784 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f85c7409ed43b7ad1badbdc4e05992', 'request_id': 'req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:28.591973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3b2fb-cd59-4ac8-90c2-7413c1a72354 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:28.603893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3b2fb-cd59-4ac8-90c2-7413c1a72354 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:28.664251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9974f5-96c8-4ce8-b179-73309d725918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:28.665209 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9974f5-96c8-4ce8-b179-73309d725918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.665492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9974f5-96c8-4ce8-b179-73309d725918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.668059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9974f5-96c8-4ce8-b179-73309d725918 admin admin] PATH_INFO: `/v3/projects/e3e2c4fb7f354179a90bfa7d963aa50c/users/8016e561397f4b4faca9e03343d51355/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.680590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9974f5-96c8-4ce8-b179-73309d725918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e2c4fb7f354179a90bfa7d963aa50c, user_id=8016e561397f4b4faca9e03343d51355, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:28.683371 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9974f5-96c8-4ce8-b179-73309d725918 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:28.712165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f85c7409ed43b7ad1badbdc4e05992', 'request_id': 'req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:28.718441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f85c7409ed43b7ad1badbdc4e05992', 'request_id': 'req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:28.731471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] The token cache is being invalidate because project 00f85c7409ed43b7ad1badbdc4e05992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:28.731767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:28.732448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5ce4095c-ba7a-4d7d-bd51-09f0646783d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:28.734206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1073/2146] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:28 2022] DELETE /identity/v3/projects/00f85c7409ed43b7ad1badbdc4e05992 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:28.735469 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1074/2147] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:28 2022] PUT /identity/v3/projects/e3e2c4fb7f354179a90bfa7d963aa50c/users/8016e561397f4b4faca9e03343d51355/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:28.738884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66020b57-f7fa-41fa-a71d-f3b8618c3f93 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.739184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66020b57-f7fa-41fa-a71d-f3b8618c3f93 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.739481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66020b57-f7fa-41fa-a71d-f3b8618c3f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.740567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ad083-4a56-4c54-8ffb-44d27e73332a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.740861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ad083-4a56-4c54-8ffb-44d27e73332a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.741164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ad083-4a56-4c54-8ffb-44d27e73332a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.758095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-66020b57-f7fa-41fa-a71d-f3b8618c3f93 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:28.758906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-338ad083-4a56-4c54-8ffb-44d27e73332a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:28.759414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-66020b57-f7fa-41fa-a71d-f3b8618c3f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:28.760143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-338ad083-4a56-4c54-8ffb-44d27e73332a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:28.807784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:28.822683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1074/2148] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:28.823035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1075/2149] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:28.830744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b7ae5-39b1-49aa-930d-5dc2091bf790 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.831096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b7ae5-39b1-49aa-930d-5dc2091bf790 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.832348 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b7ae5-39b1-49aa-930d-5dc2091bf790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.844819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ad083-4a56-4c54-8ffb-44d27e73332a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:28.846859 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-338ad083-4a56-4c54-8ffb-44d27e73332a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:28.854752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-545b7ae5-39b1-49aa-930d-5dc2091bf790 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:28.857056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-545b7ae5-39b1-49aa-930d-5dc2091bf790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:28.868906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1075/2150] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:28.877482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b7ae5-39b1-49aa-930d-5dc2091bf790 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:28.897473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b7ae5-39b1-49aa-930d-5dc2091bf790 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:28.906701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cc520-312c-4d47-b9d4-08e52731f75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:28.907394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cc520-312c-4d47-b9d4-08e52731f75a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.907684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cc520-312c-4d47-b9d4-08e52731f75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.908307 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cc520-312c-4d47-b9d4-08e52731f75a admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.908709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cc520-312c-4d47-b9d4-08e52731f75a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:28.909057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cc520-312c-4d47-b9d4-08e52731f75a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:28.910875 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cc520-312c-4d47-b9d4-08e52731f75a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:28.924013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1076/2151] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:51:28.924555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ad083-4a56-4c54-8ffb-44d27e73332a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:28.929991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cc520-312c-4d47-b9d4-08e52731f75a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:28.930369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e9832-fa4a-45ac-97a8-9842f2a60ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:28.931802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9832-fa4a-45ac-97a8-9842f2a60ffa service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.932178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9832-fa4a-45ac-97a8-9842f2a60ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.932466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e9832-fa4a-45ac-97a8-9842f2a60ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.942819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cc520-312c-4d47-b9d4-08e52731f75a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:28.946638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e9832-fa4a-45ac-97a8-9842f2a60ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:28.948105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a263d-f19c-404a-a113-9e7a66efffe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:28.948251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e9832-fa4a-45ac-97a8-9842f2a60ffa service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:28.948991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a263d-f19c-404a-a113-9e7a66efffe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:28.949386 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a263d-f19c-404a-a113-9e7a66efffe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:28.949844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a263d-f19c-404a-a113-9e7a66efffe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:28.950745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a263d-f19c-404a-a113-9e7a66efffe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:28.952957 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a263d-f19c-404a-a113-9e7a66efffe7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:28.964168 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1076/2152] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:29.081329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:29.094093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1077/2153] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:28 2022] POST /identity/v3/projects => generated 367 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:29.098802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e9832-fa4a-45ac-97a8-9842f2a60ffa service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:29.110876 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e9832-fa4a-45ac-97a8-9842f2a60ffa service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:29.171781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81ad9f-054d-423b-ac04-d5c959cab0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:29.172472 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81ad9f-054d-423b-ac04-d5c959cab0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:29.172782 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81ad9f-054d-423b-ac04-d5c959cab0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:29.173140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81ad9f-054d-423b-ac04-d5c959cab0d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:29.173931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81ad9f-054d-423b-ac04-d5c959cab0d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:29.175971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81ad9f-054d-423b-ac04-d5c959cab0d7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:29.267567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1077/2154] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:29 2022] POST /identity/v3/users => generated 284 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:29.272046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a263d-f19c-404a-a113-9e7a66efffe7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:29.282639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a263d-f19c-404a-a113-9e7a66efffe7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:29.288244 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19145a8-33a7-4880-b1a8-b1ea2b6db5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:29.289026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19145a8-33a7-4880-b1a8-b1ea2b6db5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:29.290040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19145a8-33a7-4880-b1a8-b1ea2b6db5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:29.290763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19145a8-33a7-4880-b1a8-b1ea2b6db5d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:29.291751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19145a8-33a7-4880-b1a8-b1ea2b6db5d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:29.293436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19145a8-33a7-4880-b1a8-b1ea2b6db5d2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:29.306021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1078/2155] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:29.309611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81ad9f-054d-423b-ac04-d5c959cab0d7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:29.321535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81ad9f-054d-423b-ac04-d5c959cab0d7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:29.327543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdaf0d-1045-4cf7-a7f1-309828cfaded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:29.328366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdaf0d-1045-4cf7-a7f1-309828cfaded admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:29.328656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdaf0d-1045-4cf7-a7f1-309828cfaded admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:29.328962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdaf0d-1045-4cf7-a7f1-309828cfaded admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:29.329750 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdaf0d-1045-4cf7-a7f1-309828cfaded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:29.331491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdaf0d-1045-4cf7-a7f1-309828cfaded admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:29.346628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1078/2156] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:29.347733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19145a8-33a7-4880-b1a8-b1ea2b6db5d2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:29.362538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19145a8-33a7-4880-b1a8-b1ea2b6db5d2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:29.369314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a74ee8-de1f-40a2-a2b6-82e37f542c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:29.370305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a74ee8-de1f-40a2-a2b6-82e37f542c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:29.370986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a74ee8-de1f-40a2-a2b6-82e37f542c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:29.370986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a74ee8-de1f-40a2-a2b6-82e37f542c97 admin admin] PATH_INFO: `/v3/projects/556c90cae9164495b9d370bc8cd85133/users/e470e77147ce4f088750ed66a285d697/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:29.388458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a74ee8-de1f-40a2-a2b6-82e37f542c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556c90cae9164495b9d370bc8cd85133, user_id=e470e77147ce4f088750ed66a285d697, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:29.391502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a74ee8-de1f-40a2-a2b6-82e37f542c97 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:29.476098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1079/2157] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:29 2022] PUT /identity/v3/projects/556c90cae9164495b9d370bc8cd85133/users/e470e77147ce4f088750ed66a285d697/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:29.487279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d6562-7abe-441f-a837-b0f5ed087001 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:29.487802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d6562-7abe-441f-a837-b0f5ed087001 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:29.488814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d6562-7abe-441f-a837-b0f5ed087001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:29.520280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-341d6562-7abe-441f-a837-b0f5ed087001 None None] MFA Rules not processed for user `e470e77147ce4f088750ed66a285d697`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:29.523017 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-341d6562-7abe-441f-a837-b0f5ed087001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:29.618723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1079/2158] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:29.623395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a74ee8-de1f-40a2-a2b6-82e37f542c97 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:29.647458 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d6562-7abe-441f-a837-b0f5ed087001 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:29.649588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-341d6562-7abe-441f-a837-b0f5ed087001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:29.657798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:29.739896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a74ee8-de1f-40a2-a2b6-82e37f542c97 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:29.744831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9c1ff-121c-46e2-99ca-7343845d0689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:29.745447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d6562-7abe-441f-a837-b0f5ed087001 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:29.745665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9c1ff-121c-46e2-99ca-7343845d0689 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:29.745953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9c1ff-121c-46e2-99ca-7343845d0689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:29.746358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9c1ff-121c-46e2-99ca-7343845d0689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:29.747172 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9c1ff-121c-46e2-99ca-7343845d0689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:29.749131 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9c1ff-121c-46e2-99ca-7343845d0689 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:29.751683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfa385-0b85-479d-b9d7-8fa101ab5374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:29.752690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfa385-0b85-479d-b9d7-8fa101ab5374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:29.753014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfa385-0b85-479d-b9d7-8fa101ab5374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:29.753296 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfa385-0b85-479d-b9d7-8fa101ab5374 admin admin] PATH_INFO: `/v3/users/bd11db69638945b087b842059d7e47c0` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:29.754533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfa385-0b85-479d-b9d7-8fa101ab5374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd11db69638945b087b842059d7e47c0)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:29.757032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfa385-0b85-479d-b9d7-8fa101ab5374 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:29.829377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1080/2159] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:29 2022] POST /identity/v3/projects => generated 367 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:29.833724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9c1ff-121c-46e2-99ca-7343845d0689 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:29.905188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9c1ff-121c-46e2-99ca-7343845d0689 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:29.911299 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b090-b1cb-4312-afbf-b97774cee5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:29.911982 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b090-b1cb-4312-afbf-b97774cee5ee admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:29.912277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b090-b1cb-4312-afbf-b97774cee5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:29.912554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b090-b1cb-4312-afbf-b97774cee5ee admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:29.913342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88b090-b1cb-4312-afbf-b97774cee5ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:29.915282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88b090-b1cb-4312-afbf-b97774cee5ee admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:29.967193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-50dfa385-0b85-479d-b9d7-8fa101ab5374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd11db69638945b087b842059d7e47c0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:29.976188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-50dfa385-0b85-479d-b9d7-8fa101ab5374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd11db69638945b087b842059d7e47c0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:29.990124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-50dfa385-0b85-479d-b9d7-8fa101ab5374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd11db69638945b087b842059d7e47c0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:30.034369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1081/2160] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:29 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:30.038948 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b090-b1cb-4312-afbf-b97774cee5ee admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:30.052275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b090-b1cb-4312-afbf-b97774cee5ee admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:30.056136 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-50dfa385-0b85-479d-b9d7-8fa101ab5374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd11db69638945b087b842059d7e47c0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:30.057297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-50dfa385-0b85-479d-b9d7-8fa101ab5374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd11db69638945b087b842059d7e47c0'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:30.062636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d429f-9c34-4611-90bf-f4c9fe83b355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:30.063355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d429f-9c34-4611-90bf-f4c9fe83b355 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:30.063647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d429f-9c34-4611-90bf-f4c9fe83b355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:30.063947 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d429f-9c34-4611-90bf-f4c9fe83b355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:30.064666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d429f-9c34-4611-90bf-f4c9fe83b355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:30.067648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d429f-9c34-4611-90bf-f4c9fe83b355 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:30.076192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1080/2161] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:29 2022] DELETE /identity/v3/users/bd11db69638945b087b842059d7e47c0 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:30.080839 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491f547-c065-4584-84c4-1d3aa5aba357 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:30.081212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491f547-c065-4584-84c4-1d3aa5aba357 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:30.081496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491f547-c065-4584-84c4-1d3aa5aba357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:30.084559 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1082/2162] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:30.088963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d429f-9c34-4611-90bf-f4c9fe83b355 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:30.091498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4c2d429f-9c34-4611-90bf-f4c9fe83b355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:30.104035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9491f547-c065-4584-84c4-1d3aa5aba357 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:30.105541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9491f547-c065-4584-84c4-1d3aa5aba357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:30.110122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d429f-9c34-4611-90bf-f4c9fe83b355 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:30.142455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:30.178437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1a147-f4f1-47e4-affb-7859975dabea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:30.179343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1a147-f4f1-47e4-affb-7859975dabea admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:30.179801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1a147-f4f1-47e4-affb-7859975dabea admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:30.180083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1a147-f4f1-47e4-affb-7859975dabea admin admin] PATH_INFO: `/v3/projects/aa62f581ec32489ba569f7e7826b4287/users/3620d0965019471ca0723d82d5bdf3df/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:30.181693 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1081/2163] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:30.191608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491f547-c065-4584-84c4-1d3aa5aba357 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:30.193167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9491f547-c065-4584-84c4-1d3aa5aba357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:30.195149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1a147-f4f1-47e4-affb-7859975dabea admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa62f581ec32489ba569f7e7826b4287, user_id=3620d0965019471ca0723d82d5bdf3df, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:30.197985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea1a147-f4f1-47e4-affb-7859975dabea admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:30.254305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491f547-c065-4584-84c4-1d3aa5aba357 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:30.259365 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e21a9-95ca-4d57-99bc-790b9ca04df4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:30.259365 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e21a9-95ca-4d57-99bc-790b9ca04df4 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:30.259655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e21a9-95ca-4d57-99bc-790b9ca04df4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:30.260010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e21a9-95ca-4d57-99bc-790b9ca04df4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:30.270646 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e21a9-95ca-4d57-99bc-790b9ca04df4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:30.272158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e21a9-95ca-4d57-99bc-790b9ca04df4 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:30.282539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1082/2164] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:30.283676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1083/2165] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:30 2022] PUT /identity/v3/projects/aa62f581ec32489ba569f7e7826b4287/users/3620d0965019471ca0723d82d5bdf3df/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:30.288284 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e21a9-95ca-4d57-99bc-790b9ca04df4 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:30.297094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e21a9-95ca-4d57-99bc-790b9ca04df4 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:30.373844 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d10ab-d5e4-4bd6-9ab9-8b1605a2fa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:30.374580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d10ab-d5e4-4bd6-9ab9-8b1605a2fa0d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:30.374866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d10ab-d5e4-4bd6-9ab9-8b1605a2fa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:30.375141 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d10ab-d5e4-4bd6-9ab9-8b1605a2fa0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:30.375883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d10ab-d5e4-4bd6-9ab9-8b1605a2fa0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:30.377551 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d10ab-d5e4-4bd6-9ab9-8b1605a2fa0d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:30.388655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1083/2166] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:30.393857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1a147-f4f1-47e4-affb-7859975dabea admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:30.408812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1a147-f4f1-47e4-affb-7859975dabea admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:30.415109 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e0d79-9e24-4562-88aa-f53aa55a22ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:30.416115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e0d79-9e24-4562-88aa-f53aa55a22ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:30.416340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e0d79-9e24-4562-88aa-f53aa55a22ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:30.416617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e0d79-9e24-4562-88aa-f53aa55a22ba admin admin] PATH_INFO: `/v3/projects/aa62f581ec32489ba569f7e7826b4287/users/3620d0965019471ca0723d82d5bdf3df/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:30.419165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e0d79-9e24-4562-88aa-f53aa55a22ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa62f581ec32489ba569f7e7826b4287, user_id=3620d0965019471ca0723d82d5bdf3df, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:30.422036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e0d79-9e24-4562-88aa-f53aa55a22ba admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:30.434623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1084/2167] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:30 2022] PUT /identity/v3/projects/aa62f581ec32489ba569f7e7826b4287/users/3620d0965019471ca0723d82d5bdf3df/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:30.449844 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffff7d8-0b78-4b9d-971b-dabd3a0f5304 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:30.450050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffff7d8-0b78-4b9d-971b-dabd3a0f5304 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:30.450300 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffff7d8-0b78-4b9d-971b-dabd3a0f5304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:30.467132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:30.480385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7ffff7d8-0b78-4b9d-971b-dabd3a0f5304 None None] MFA Rules not processed for user `3620d0965019471ca0723d82d5bdf3df`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:30.481707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7ffff7d8-0b78-4b9d-971b-dabd3a0f5304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:30.598546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1084/2168] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:30.611794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e0d79-9e24-4562-88aa-f53aa55a22ba admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:30.614469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-569e0d79-9e24-4562-88aa-f53aa55a22ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:30.686336 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e0d79-9e24-4562-88aa-f53aa55a22ba admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:30.690341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32d206-d919-420a-b496-5745dd30657d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:30.691016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32d206-d919-420a-b496-5745dd30657d service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:30.691301 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32d206-d919-420a-b496-5745dd30657d service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:30.691578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32d206-d919-420a-b496-5745dd30657d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:30.704705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32d206-d919-420a-b496-5745dd30657d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:30.705922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32d206-d919-420a-b496-5745dd30657d service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:30.719477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1085/2169] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:30.945632 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffff7d8-0b78-4b9d-971b-dabd3a0f5304 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:30.949815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7ffff7d8-0b78-4b9d-971b-dabd3a0f5304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:30.976655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32d206-d919-420a-b496-5745dd30657d service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:30.986305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32d206-d919-420a-b496-5745dd30657d service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:31.000467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b569ef0-5548-466c-a940-0a5a16fa9cc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:31.000467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b569ef0-5548-466c-a940-0a5a16fa9cc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:31.000467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b569ef0-5548-466c-a940-0a5a16fa9cc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:31.000467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b569ef0-5548-466c-a940-0a5a16fa9cc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:31.005253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6b569ef0-5548-466c-a940-0a5a16fa9cc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:31.099073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffff7d8-0b78-4b9d-971b-dabd3a0f5304 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:31.105878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:31.107010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:31.107369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:31.107722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] PATH_INFO: `/v3/projects/d62c01d6a8ad4051b60c4a95a1ce13ae` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:31.109462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62c01d6a8ad4051b60c4a95a1ce13ae)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:31.111482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:31.145429 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b569ef0-5548-466c-a940-0a5a16fa9cc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:31.146903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b569ef0-5548-466c-a940-0a5a16fa9cc1 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:31.167301 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1086/2170] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:31.298706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62c01d6a8ad4051b60c4a95a1ce13ae', 'request_id': 'req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:31.307909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62c01d6a8ad4051b60c4a95a1ce13ae', 'request_id': 'req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:31.313766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62c01d6a8ad4051b60c4a95a1ce13ae', 'request_id': 'req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:31.428199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] The token cache is being invalidate because project d62c01d6a8ad4051b60c4a95a1ce13ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:31.428792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:31.433191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:31.436707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1085/2171] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:30 2022] DELETE /identity/v3/projects/d62c01d6a8ad4051b60c4a95a1ce13ae => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:31.441833 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b569ef0-5548-466c-a940-0a5a16fa9cc1 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:31.452690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b569ef0-5548-466c-a940-0a5a16fa9cc1 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:31.527531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:31.549516 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:31.551158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:31.551446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:31.551752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] PATH_INFO: `/v3/users/08fb4c16a8b44c299939141c3308a7c3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:31.553535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=08fb4c16a8b44c299939141c3308a7c3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:31.555630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:31.808237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08fb4c16a8b44c299939141c3308a7c3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:31.832464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08fb4c16a8b44c299939141c3308a7c3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:31.841118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08fb4c16a8b44c299939141c3308a7c3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:31.943931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08fb4c16a8b44c299939141c3308a7c3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:31.956227 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08fb4c16a8b44c299939141c3308a7c3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:31.962359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1087/2172] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:31 2022] DELETE /identity/v3/users/08fb4c16a8b44c299939141c3308a7c3 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:32.526159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:32.816007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:32.818050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:32.834890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eb75a-dcd7-4b5b-b281-eebf396eaf90 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:32.835041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:32.837069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:32.946934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:32.947936 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:32.948327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:32.948871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] PATH_INFO: `/v3/projects/cd9e1e82103248398547480c762d59a9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:32.950519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd9e1e82103248398547480c762d59a9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:32.955649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:32.976469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347398e-05ec-4389-998e-6f0307bee9ce admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:32.982099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18c054-62fb-4e45-968c-2fd2ef43b3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:32.983501 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18c054-62fb-4e45-968c-2fd2ef43b3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:32.984340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18c054-62fb-4e45-968c-2fd2ef43b3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:32.984456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18c054-62fb-4e45-968c-2fd2ef43b3b0 admin admin] PATH_INFO: `/v3/users/9431cd1afee04d408bceeb353b263241` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:32.986949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18c054-62fb-4e45-968c-2fd2ef43b3b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9431cd1afee04d408bceeb353b263241)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:32.990590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18c054-62fb-4e45-968c-2fd2ef43b3b0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:33.130434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd9e1e82103248398547480c762d59a9', 'request_id': 'req-537869eb-4624-4f74-bd2e-4007d0425a62'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:33.139324 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd9e1e82103248398547480c762d59a9', 'request_id': 'req-537869eb-4624-4f74-bd2e-4007d0425a62'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:33.169349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd9e1e82103248398547480c762d59a9', 'request_id': 'req-537869eb-4624-4f74-bd2e-4007d0425a62'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:33.214011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bd18c054-62fb-4e45-968c-2fd2ef43b3b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9431cd1afee04d408bceeb353b263241'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:33.239481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] The token cache is being invalidate because project cd9e1e82103248398547480c762d59a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:33.241913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:33.244934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-537869eb-4624-4f74-bd2e-4007d0425a62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:33.247092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1086/2173] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:32 2022] DELETE /identity/v3/projects/cd9e1e82103248398547480c762d59a9 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:33.261830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b97be-d387-4736-8b69-7c728084fa0c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:33.262152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b97be-d387-4736-8b69-7c728084fa0c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:33.262440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b97be-d387-4736-8b69-7c728084fa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:33.276062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:33.279083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bd18c054-62fb-4e45-968c-2fd2ef43b3b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9431cd1afee04d408bceeb353b263241'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:33.290070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bd18c054-62fb-4e45-968c-2fd2ef43b3b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9431cd1afee04d408bceeb353b263241'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:33.301125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:33.313979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-109b97be-d387-4736-8b69-7c728084fa0c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:33.317304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-109b97be-d387-4736-8b69-7c728084fa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:33.426762 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bd18c054-62fb-4e45-968c-2fd2ef43b3b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9431cd1afee04d408bceeb353b263241'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:33.427700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bd18c054-62fb-4e45-968c-2fd2ef43b3b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9431cd1afee04d408bceeb353b263241'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:33.436109 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1088/2174] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:32 2022] DELETE /identity/v3/users/9431cd1afee04d408bceeb353b263241 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:33.442550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab8a44-543c-4427-8c67-3a66500ed56a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:33.443212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab8a44-543c-4427-8c67-3a66500ed56a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:33.443668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab8a44-543c-4427-8c67-3a66500ed56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:33.461978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1087/2175] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:33.463208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c77c9e-322c-4f15-a717-ea9fadf8e3f9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:33.463514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c77c9e-322c-4f15-a717-ea9fadf8e3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:33.463794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c77c9e-322c-4f15-a717-ea9fadf8e3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:33.496949 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-75c77c9e-322c-4f15-a717-ea9fadf8e3f9 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:33.498274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-75c77c9e-322c-4f15-a717-ea9fadf8e3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:33.504641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-59ab8a44-543c-4427-8c67-3a66500ed56a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:33.506191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-59ab8a44-543c-4427-8c67-3a66500ed56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:33.630237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1088/2176] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:33.635030 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c77c9e-322c-4f15-a717-ea9fadf8e3f9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:33.642516 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1089/2177] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:33.649461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab8a44-543c-4427-8c67-3a66500ed56a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:33.653183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-59ab8a44-543c-4427-8c67-3a66500ed56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:33.662733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c77c9e-322c-4f15-a717-ea9fadf8e3f9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:33.674239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c679196-1c87-41af-ae81-aa2837a80e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:33.675979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c679196-1c87-41af-ae81-aa2837a80e04 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:33.676279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c679196-1c87-41af-ae81-aa2837a80e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:33.676561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c679196-1c87-41af-ae81-aa2837a80e04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:33.677391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c679196-1c87-41af-ae81-aa2837a80e04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:33.677880 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c679196-1c87-41af-ae81-aa2837a80e04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:33.679623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c679196-1c87-41af-ae81-aa2837a80e04 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:33.699643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1089/2178] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:33.704610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c679196-1c87-41af-ae81-aa2837a80e04 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:33.726278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c679196-1c87-41af-ae81-aa2837a80e04 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:33.732324 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd241f01-1168-4825-93c7-b291e8e40323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:33.733246 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd241f01-1168-4825-93c7-b291e8e40323 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:33.733679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd241f01-1168-4825-93c7-b291e8e40323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:33.734104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd241f01-1168-4825-93c7-b291e8e40323 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:33.739997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd241f01-1168-4825-93c7-b291e8e40323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:33.739997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd241f01-1168-4825-93c7-b291e8e40323 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:33.792308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab8a44-543c-4427-8c67-3a66500ed56a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:33.796434 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307924a2-24d6-41d5-92c5-be6026e15dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:33.797241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307924a2-24d6-41d5-92c5-be6026e15dca service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:33.797531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307924a2-24d6-41d5-92c5-be6026e15dca service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:33.797814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307924a2-24d6-41d5-92c5-be6026e15dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:33.809807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307924a2-24d6-41d5-92c5-be6026e15dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:33.812193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307924a2-24d6-41d5-92c5-be6026e15dca service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:33.824574 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1090/2179] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:33.852666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1090/2180] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:33 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:33.856670 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307924a2-24d6-41d5-92c5-be6026e15dca service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:33.877414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307924a2-24d6-41d5-92c5-be6026e15dca service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:33.989777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e3d5c-8a96-4398-8d61-319ec84abc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:33.991025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e3d5c-8a96-4398-8d61-319ec84abc50 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:33.991334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e3d5c-8a96-4398-8d61-319ec84abc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:33.991623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e3d5c-8a96-4398-8d61-319ec84abc50 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:33.992344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e3d5c-8a96-4398-8d61-319ec84abc50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:33.995293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e3d5c-8a96-4398-8d61-319ec84abc50 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:34.095554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1091/2181] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:33 2022] POST /identity/v3/users => generated 290 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:34.103153 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd241f01-1168-4825-93c7-b291e8e40323 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:34.118913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd241f01-1168-4825-93c7-b291e8e40323 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:34.129284 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999a240-8f4d-4a85-ae63-d49f3366e1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:34.129975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999a240-8f4d-4a85-ae63-d49f3366e1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:34.130268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999a240-8f4d-4a85-ae63-d49f3366e1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:34.130537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999a240-8f4d-4a85-ae63-d49f3366e1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:34.131279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999a240-8f4d-4a85-ae63-d49f3366e1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:34.137939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999a240-8f4d-4a85-ae63-d49f3366e1b8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:34.166392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1091/2182] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:34.166800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e3d5c-8a96-4398-8d61-319ec84abc50 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:34.184331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e3d5c-8a96-4398-8d61-319ec84abc50 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:34.195961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1184-d469-4511-9a1b-bd037aa71600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:34.196732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1184-d469-4511-9a1b-bd037aa71600 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:34.197061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1184-d469-4511-9a1b-bd037aa71600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:34.197342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1184-d469-4511-9a1b-bd037aa71600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:34.198098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc1184-d469-4511-9a1b-bd037aa71600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:34.207772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc1184-d469-4511-9a1b-bd037aa71600 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:34.228476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1092/2183] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:34.230516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999a240-8f4d-4a85-ae63-d49f3366e1b8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:34.243649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999a240-8f4d-4a85-ae63-d49f3366e1b8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:34.248495 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869763a3-db18-4781-9396-02a8a7b335d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:34.249494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869763a3-db18-4781-9396-02a8a7b335d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:34.249965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869763a3-db18-4781-9396-02a8a7b335d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:34.250234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869763a3-db18-4781-9396-02a8a7b335d0 admin admin] PATH_INFO: `/v3/projects/2fcd784fd30e41629dd3449eb4969e34/users/55e8b1fa83fb4e9593dc60735feeb01b/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:34.267408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869763a3-db18-4781-9396-02a8a7b335d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fcd784fd30e41629dd3449eb4969e34, user_id=55e8b1fa83fb4e9593dc60735feeb01b, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:34.278627 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869763a3-db18-4781-9396-02a8a7b335d0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:34.325191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1184-d469-4511-9a1b-bd037aa71600 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:34.327074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eddc1184-d469-4511-9a1b-bd037aa71600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:34.347962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1184-d469-4511-9a1b-bd037aa71600 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:34.356756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:34.357748 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:34.358030 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:34.358312 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] PATH_INFO: `/v3/projects/3859defa71b445aca9597bbeba642819` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:34.359606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3859defa71b445aca9597bbeba642819)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:34.361874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:34.412114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1092/2184] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:34 2022] PUT /identity/v3/projects/2fcd784fd30e41629dd3449eb4969e34/users/55e8b1fa83fb4e9593dc60735feeb01b/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:34.422844 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43c814-ec80-44a5-8701-5ab2610b0fa4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:34.423253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43c814-ec80-44a5-8701-5ab2610b0fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:34.423676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43c814-ec80-44a5-8701-5ab2610b0fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:34.483832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3c43c814-ec80-44a5-8701-5ab2610b0fa4 None None] MFA Rules not processed for user `55e8b1fa83fb4e9593dc60735feeb01b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:34.485471 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3c43c814-ec80-44a5-8701-5ab2610b0fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:34.533620 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3859defa71b445aca9597bbeba642819', 'request_id': 'req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:34.592231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1093/2185] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:34.597148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43c814-ec80-44a5-8701-5ab2610b0fa4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:34.640542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3859defa71b445aca9597bbeba642819', 'request_id': 'req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:34.661189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3859defa71b445aca9597bbeba642819', 'request_id': 'req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:34.684141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] The token cache is being invalidate because project 3859defa71b445aca9597bbeba642819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:34.684686 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:34.686325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:34.688910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1093/2186] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:34 2022] DELETE /identity/v3/projects/3859defa71b445aca9597bbeba642819 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:34.711894 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:34.722012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:34.722158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43c814-ec80-44a5-8701-5ab2610b0fa4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:34.828396 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46650398-eca6-410a-94df-1ded172eac0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:34.828595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b3fb2-b158-4ab0-a4d2-f30a0c0750a9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:34.830987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46650398-eca6-410a-94df-1ded172eac0d admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:34.831285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46650398-eca6-410a-94df-1ded172eac0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:34.831604 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46650398-eca6-410a-94df-1ded172eac0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:34.832331 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46650398-eca6-410a-94df-1ded172eac0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:34.834404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:34.835373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:34.835663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:34.835941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] PATH_INFO: `/v3/users/7c43890dd4554c9fa80346ffaf3f910c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:34.836263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46650398-eca6-410a-94df-1ded172eac0d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:34.837702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c43890dd4554c9fa80346ffaf3f910c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:34.839995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:34.952595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1094/2187] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:34 2022] POST /identity/v3/projects => generated 377 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:34.958221 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46650398-eca6-410a-94df-1ded172eac0d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:34.962543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-46650398-eca6-410a-94df-1ded172eac0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:34.969209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:35.082829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c43890dd4554c9fa80346ffaf3f910c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:35.094951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46650398-eca6-410a-94df-1ded172eac0d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:35.106788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c43890dd4554c9fa80346ffaf3f910c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:35.108631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4f4c3-81b3-4e62-b887-f00432251828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:35.109441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4f4c3-81b3-4e62-b887-f00432251828 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:35.109665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4f4c3-81b3-4e62-b887-f00432251828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:35.111333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4f4c3-81b3-4e62-b887-f00432251828 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:35.112149 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4f4c3-81b3-4e62-b887-f00432251828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:35.114749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c43890dd4554c9fa80346ffaf3f910c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:35.118136 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4f4c3-81b3-4e62-b887-f00432251828 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:35.165811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c43890dd4554c9fa80346ffaf3f910c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:35.167339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c43890dd4554c9fa80346ffaf3f910c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:35.187100 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1094/2188] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:34 2022] DELETE /identity/v3/users/7c43890dd4554c9fa80346ffaf3f910c => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:35.251402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1095/2189] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:34 2022] POST /identity/v3/users => generated 289 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:35.255465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:35.257196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:35.329266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:35.382443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398e6f4-5a6c-4ba6-aef1-82a2029cfc36 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:35.388789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890bbc8-eb77-464c-b3ef-4e440608d565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:35.389597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890bbc8-eb77-464c-b3ef-4e440608d565 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:35.389983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890bbc8-eb77-464c-b3ef-4e440608d565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:35.390341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890bbc8-eb77-464c-b3ef-4e440608d565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:35.391291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890bbc8-eb77-464c-b3ef-4e440608d565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:35.393204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890bbc8-eb77-464c-b3ef-4e440608d565 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:35.405452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1095/2190] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:35.409527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4f4c3-81b3-4e62-b887-f00432251828 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:35.429201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4f4c3-81b3-4e62-b887-f00432251828 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:35.436477 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d00326-12e4-474e-ace0-5a9076f3b53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:35.437380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d00326-12e4-474e-ace0-5a9076f3b53c admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:35.437673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d00326-12e4-474e-ace0-5a9076f3b53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:35.438099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d00326-12e4-474e-ace0-5a9076f3b53c admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:35.438774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d00326-12e4-474e-ace0-5a9076f3b53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:35.440451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d00326-12e4-474e-ace0-5a9076f3b53c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:35.453911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1096/2191] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:35.458404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890bbc8-eb77-464c-b3ef-4e440608d565 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:35.472867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890bbc8-eb77-464c-b3ef-4e440608d565 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:35.479120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799387f-e7fc-4e6d-ad0b-3cb7ac8e7519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:35.480640 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799387f-e7fc-4e6d-ad0b-3cb7ac8e7519 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:35.481065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799387f-e7fc-4e6d-ad0b-3cb7ac8e7519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:35.481525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799387f-e7fc-4e6d-ad0b-3cb7ac8e7519 admin admin] PATH_INFO: `/v3/projects/05dbb1af9525455eb1fc6888e965de49/users/7ba145b5874c4f2ca3d9e4ec1c328181/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:35.524151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799387f-e7fc-4e6d-ad0b-3cb7ac8e7519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05dbb1af9525455eb1fc6888e965de49, user_id=7ba145b5874c4f2ca3d9e4ec1c328181, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:35.531434 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799387f-e7fc-4e6d-ad0b-3cb7ac8e7519 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:35.657337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1096/2192] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:35 2022] PUT /identity/v3/projects/05dbb1af9525455eb1fc6888e965de49/users/7ba145b5874c4f2ca3d9e4ec1c328181/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:35.673877 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4d507-8f68-4df2-aa21-26aee766af60 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:35.674181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4d507-8f68-4df2-aa21-26aee766af60 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:35.674451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4d507-8f68-4df2-aa21-26aee766af60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:35.734972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d0f4d507-8f68-4df2-aa21-26aee766af60 None None] MFA Rules not processed for user `7ba145b5874c4f2ca3d9e4ec1c328181`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:35.739172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d0f4d507-8f68-4df2-aa21-26aee766af60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:35.863698 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799387f-e7fc-4e6d-ad0b-3cb7ac8e7519 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:35.872162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0799387f-e7fc-4e6d-ad0b-3cb7ac8e7519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:35.937971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1097/2193] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:35.951772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4d507-8f68-4df2-aa21-26aee766af60 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:36.038442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799387f-e7fc-4e6d-ad0b-3cb7ac8e7519 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.046995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.048054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.050079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.050079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] PATH_INFO: `/v3/projects/d0ad9f0355b24cfaaf1ea42eaae04832` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.050563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ad9f0355b24cfaaf1ea42eaae04832)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.052733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.061526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4d507-8f68-4df2-aa21-26aee766af60 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.067784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32f1c6-2fbb-4771-9533-0deb704702a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.068470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32f1c6-2fbb-4771-9533-0deb704702a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.068759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32f1c6-2fbb-4771-9533-0deb704702a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.069250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32f1c6-2fbb-4771-9533-0deb704702a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.069967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32f1c6-2fbb-4771-9533-0deb704702a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.072191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32f1c6-2fbb-4771-9533-0deb704702a4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.166203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1098/2194] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:35 2022] POST /identity/v3/projects => generated 377 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:36.171308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32f1c6-2fbb-4771-9533-0deb704702a4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:36.202427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ad9f0355b24cfaaf1ea42eaae04832', 'request_id': 'req-81829c97-6dfc-438d-b325-254468b959e4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:36.298580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32f1c6-2fbb-4771-9533-0deb704702a4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.318876 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ad9f0355b24cfaaf1ea42eaae04832', 'request_id': 'req-81829c97-6dfc-438d-b325-254468b959e4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:36.333522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ad9f0355b24cfaaf1ea42eaae04832', 'request_id': 'req-81829c97-6dfc-438d-b325-254468b959e4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:36.358964 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] The token cache is being invalidate because project d0ad9f0355b24cfaaf1ea42eaae04832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:36.371352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:36.372248 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81829c97-6dfc-438d-b325-254468b959e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:36.376423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1097/2195] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:35 2022] DELETE /identity/v3/projects/d0ad9f0355b24cfaaf1ea42eaae04832 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:36.386580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c7521-c922-486e-9e47-c1fcc7ad3081 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.386580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c7521-c922-486e-9e47-c1fcc7ad3081 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.386580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c7521-c922-486e-9e47-c1fcc7ad3081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.433795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-429c7521-c922-486e-9e47-c1fcc7ad3081 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:36.435506 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2870e2e-7154-4175-ac3f-3dc8b6190dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.436270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870e2e-7154-4175-ac3f-3dc8b6190dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.436855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870e2e-7154-4175-ac3f-3dc8b6190dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.437227 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2870e2e-7154-4175-ac3f-3dc8b6190dcc admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.442235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2870e2e-7154-4175-ac3f-3dc8b6190dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.444565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-429c7521-c922-486e-9e47-c1fcc7ad3081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:36.446013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2870e2e-7154-4175-ac3f-3dc8b6190dcc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.465502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1098/2196] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:36.474935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e3da8-97a2-4970-b122-04a94ba50bf6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.475420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e3da8-97a2-4970-b122-04a94ba50bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.475579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e3da8-97a2-4970-b122-04a94ba50bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.500745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ed4e3da8-97a2-4970-b122-04a94ba50bf6 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:36.502328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ed4e3da8-97a2-4970-b122-04a94ba50bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:36.511612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1099/2197] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:36.515849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e3da8-97a2-4970-b122-04a94ba50bf6 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:36.525857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e3da8-97a2-4970-b122-04a94ba50bf6 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.540003 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1099/2198] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:36 2022] POST /identity/v3/users => generated 295 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:36.541113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2870e2e-7154-4175-ac3f-3dc8b6190dcc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:36.543226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d2870e2e-7154-4175-ac3f-3dc8b6190dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:36.553222 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e5d3-b39e-4649-85bb-d7ee76d6f58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.553925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e5d3-b39e-4649-85bb-d7ee76d6f58b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.554215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e5d3-b39e-4649-85bb-d7ee76d6f58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.554497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e5d3-b39e-4649-85bb-d7ee76d6f58b admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.555260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701e5d3-b39e-4649-85bb-d7ee76d6f58b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:36.558912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701e5d3-b39e-4649-85bb-d7ee76d6f58b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.561104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701e5d3-b39e-4649-85bb-d7ee76d6f58b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.565382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2870e2e-7154-4175-ac3f-3dc8b6190dcc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.572372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eaaa5-8799-456b-8dd9-fd7a4177a6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.573315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eaaa5-8799-456b-8dd9-fd7a4177a6fc admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.574697 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eaaa5-8799-456b-8dd9-fd7a4177a6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.575984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eaaa5-8799-456b-8dd9-fd7a4177a6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.579087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4eaaa5-8799-456b-8dd9-fd7a4177a6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.581335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4eaaa5-8799-456b-8dd9-fd7a4177a6fc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.596716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1100/2199] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:36.602794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e5d3-b39e-4649-85bb-d7ee76d6f58b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:36.615401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1100/2200] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:36.620066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eaaa5-8799-456b-8dd9-fd7a4177a6fc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:36.629299 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e5d3-b39e-4649-85bb-d7ee76d6f58b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.641637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eaaa5-8799-456b-8dd9-fd7a4177a6fc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.641994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c9706-303d-4b92-bd94-f75439b91e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.642831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c9706-303d-4b92-bd94-f75439b91e1f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.643820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c9706-303d-4b92-bd94-f75439b91e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.643820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c9706-303d-4b92-bd94-f75439b91e1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.644108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c9706-303d-4b92-bd94-f75439b91e1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.646377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c9706-303d-4b92-bd94-f75439b91e1f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.650071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad1a81-b921-4075-bc93-819e1d739918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.652082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad1a81-b921-4075-bc93-819e1d739918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.652495 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad1a81-b921-4075-bc93-819e1d739918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.653104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad1a81-b921-4075-bc93-819e1d739918 admin admin] PATH_INFO: `/v3/projects/6aaca002713b437b85a40ddeb8351c4f/users/36450aa5e6e94c4f97b75e60fc726724/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.671369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad1a81-b921-4075-bc93-819e1d739918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aaca002713b437b85a40ddeb8351c4f, user_id=36450aa5e6e94c4f97b75e60fc726724, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.675622 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad1a81-b921-4075-bc93-819e1d739918 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.734687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:36.767702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1101/2201] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:36 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:36.775318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c9706-303d-4b92-bd94-f75439b91e1f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:36.783837 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1101/2202] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:36 2022] PUT /identity/v3/projects/6aaca002713b437b85a40ddeb8351c4f/users/36450aa5e6e94c4f97b75e60fc726724/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:36.787835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad1a81-b921-4075-bc93-819e1d739918 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:36.866360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad1a81-b921-4075-bc93-819e1d739918 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.869209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c9706-303d-4b92-bd94-f75439b91e1f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.872802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da3aaf-5460-44ca-8221-c134053ff9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.873776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da3aaf-5460-44ca-8221-c134053ff9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.874062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da3aaf-5460-44ca-8221-c134053ff9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.874493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da3aaf-5460-44ca-8221-c134053ff9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.876414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da3aaf-5460-44ca-8221-c134053ff9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.878751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da3aaf-5460-44ca-8221-c134053ff9e5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.878898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ea009-64ed-412a-8223-e82984cb0bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.879488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea009-64ed-412a-8223-e82984cb0bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.879947 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea009-64ed-412a-8223-e82984cb0bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.880228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ea009-64ed-412a-8223-e82984cb0bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.881142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ea009-64ed-412a-8223-e82984cb0bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.883577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ea009-64ed-412a-8223-e82984cb0bc1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.888184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:36.901302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1102/2203] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:36.902977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da3aaf-5460-44ca-8221-c134053ff9e5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:36.918681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da3aaf-5460-44ca-8221-c134053ff9e5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:36.924057 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cea7a4-1f1c-4e12-944f-88cbbaa712a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:36.925031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cea7a4-1f1c-4e12-944f-88cbbaa712a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.925393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cea7a4-1f1c-4e12-944f-88cbbaa712a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.925596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cea7a4-1f1c-4e12-944f-88cbbaa712a5 admin admin] PATH_INFO: `/v3/projects/6aaca002713b437b85a40ddeb8351c4f/users/36450aa5e6e94c4f97b75e60fc726724/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.927871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cea7a4-1f1c-4e12-944f-88cbbaa712a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aaca002713b437b85a40ddeb8351c4f, user_id=36450aa5e6e94c4f97b75e60fc726724, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:36.930617 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cea7a4-1f1c-4e12-944f-88cbbaa712a5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:36.942956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1103/2204] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:36 2022] PUT /identity/v3/projects/6aaca002713b437b85a40ddeb8351c4f/users/36450aa5e6e94c4f97b75e60fc726724/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:36.954547 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b008dc-f766-4f84-82f4-a71d678b4a4e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:36.954840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b008dc-f766-4f84-82f4-a71d678b4a4e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:36.955124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b008dc-f766-4f84-82f4-a71d678b4a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:36.981090 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1102/2205] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:36 2022] POST /identity/v3/users => generated 287 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:36.985491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ea009-64ed-412a-8223-e82984cb0bc1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:37.006152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b7b008dc-f766-4f84-82f4-a71d678b4a4e None None] MFA Rules not processed for user `36450aa5e6e94c4f97b75e60fc726724`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:37.008289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b7b008dc-f766-4f84-82f4-a71d678b4a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:37.021902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ea009-64ed-412a-8223-e82984cb0bc1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:37.167198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb32c6-4e15-43da-8ef2-37656bbf7604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:37.177998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb32c6-4e15-43da-8ef2-37656bbf7604 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:37.177998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb32c6-4e15-43da-8ef2-37656bbf7604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:37.177998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb32c6-4e15-43da-8ef2-37656bbf7604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:37.177998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb32c6-4e15-43da-8ef2-37656bbf7604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:37.177998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb32c6-4e15-43da-8ef2-37656bbf7604 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:37.186640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1104/2206] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:37.201893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1103/2207] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:37.202882 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb32c6-4e15-43da-8ef2-37656bbf7604 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:37.209445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135a434-b49a-4c26-a679-d642833c7783 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:37.213429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135a434-b49a-4c26-a679-d642833c7783 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:37.213896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135a434-b49a-4c26-a679-d642833c7783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:37.214865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb32c6-4e15-43da-8ef2-37656bbf7604 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:37.221658 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a911dc7-9215-43c8-861c-5a2c9284013c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:37.222617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911dc7-9215-43c8-861c-5a2c9284013c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:37.223054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911dc7-9215-43c8-861c-5a2c9284013c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:37.223509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911dc7-9215-43c8-861c-5a2c9284013c admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:37.224912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a911dc7-9215-43c8-861c-5a2c9284013c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:37.227777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a911dc7-9215-43c8-861c-5a2c9284013c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:37.243554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1104/2208] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:37.245578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a911dc7-9215-43c8-861c-5a2c9284013c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:37.252192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3a911dc7-9215-43c8-861c-5a2c9284013c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:37.271662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5135a434-b49a-4c26-a679-d642833c7783 None None] MFA Rules not processed for user `8016e561397f4b4faca9e03343d51355`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:37.273419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5135a434-b49a-4c26-a679-d642833c7783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:37.396167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1105/2209] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:37.398243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135a434-b49a-4c26-a679-d642833c7783 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:37.414681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135a434-b49a-4c26-a679-d642833c7783 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:37.423214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a911dc7-9215-43c8-861c-5a2c9284013c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:37.423439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff267c94-30ff-4b8f-a5f4-3915a82d9414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:37.423597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05658b-2591-4c55-a754-3f509669fc14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:37.423755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff267c94-30ff-4b8f-a5f4-3915a82d9414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:37.423969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff267c94-30ff-4b8f-a5f4-3915a82d9414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:37.424641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff267c94-30ff-4b8f-a5f4-3915a82d9414 admin admin] PATH_INFO: `/v3/projects/fd163444592a491cb81ee91021815fcf/users/4dacbeaa7ff34e5bbf95d3ac5f67a049/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:37.425044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05658b-2591-4c55-a754-3f509669fc14 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:37.425482 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05658b-2591-4c55-a754-3f509669fc14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:37.425913 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05658b-2591-4c55-a754-3f509669fc14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:37.464251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff267c94-30ff-4b8f-a5f4-3915a82d9414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd163444592a491cb81ee91021815fcf, user_id=4dacbeaa7ff34e5bbf95d3ac5f67a049, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:37.467253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff267c94-30ff-4b8f-a5f4-3915a82d9414 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:37.470825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05658b-2591-4c55-a754-3f509669fc14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:37.471872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:37.472619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05658b-2591-4c55-a754-3f509669fc14 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:37.490622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1105/2210] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:37.491078 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05658b-2591-4c55-a754-3f509669fc14 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:37.492747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ad05658b-2591-4c55-a754-3f509669fc14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:37.516062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05658b-2591-4c55-a754-3f509669fc14 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:37.522708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c179f-3c96-47e9-b941-55b989e99fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:37.524099 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c179f-3c96-47e9-b941-55b989e99fad admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:37.524453 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c179f-3c96-47e9-b941-55b989e99fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:37.524740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c179f-3c96-47e9-b941-55b989e99fad admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:37.526229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c179f-3c96-47e9-b941-55b989e99fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:37.528641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c179f-3c96-47e9-b941-55b989e99fad admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:37.601609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1106/2211] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:37 2022] PUT /identity/v3/projects/fd163444592a491cb81ee91021815fcf/users/4dacbeaa7ff34e5bbf95d3ac5f67a049/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:37.624945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff267c94-30ff-4b8f-a5f4-3915a82d9414 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:37.651095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1106/2212] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:37 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:37.658443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c179f-3c96-47e9-b941-55b989e99fad admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:37.745345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff267c94-30ff-4b8f-a5f4-3915a82d9414 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:37.794162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c179f-3c96-47e9-b941-55b989e99fad admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:37.805766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb078438-1fc8-430c-82c7-4893a774b520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:37.807197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb078438-1fc8-430c-82c7-4893a774b520 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:37.807335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb078438-1fc8-430c-82c7-4893a774b520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:37.807672 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb078438-1fc8-430c-82c7-4893a774b520 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:37.808539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb078438-1fc8-430c-82c7-4893a774b520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:37.810665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb078438-1fc8-430c-82c7-4893a774b520 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:37.830517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75544aed-c50b-455f-bd0c-45a388e62559 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:37.831808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75544aed-c50b-455f-bd0c-45a388e62559 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:37.832103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75544aed-c50b-455f-bd0c-45a388e62559 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:37.832386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75544aed-c50b-455f-bd0c-45a388e62559 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:37.837141 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-75544aed-c50b-455f-bd0c-45a388e62559 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:37.915160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75544aed-c50b-455f-bd0c-45a388e62559 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:37.916780 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75544aed-c50b-455f-bd0c-45a388e62559 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:37.945898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1107/2213] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:37.947454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751cdb3-7dc8-4d95-86c7-ffb83eb5c802 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:37.947744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751cdb3-7dc8-4d95-86c7-ffb83eb5c802 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:37.948025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751cdb3-7dc8-4d95-86c7-ffb83eb5c802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:37.951898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1107/2214] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:37 2022] POST /identity/v3/users => generated 293 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:37.955027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb078438-1fc8-430c-82c7-4893a774b520 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:37.976080 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb078438-1fc8-430c-82c7-4893a774b520 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:37.981316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a1528-95e3-47ea-a656-323b8c81e5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:37.982215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a1528-95e3-47ea-a656-323b8c81e5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:37.982651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a1528-95e3-47ea-a656-323b8c81e5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:37.983107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a1528-95e3-47ea-a656-323b8c81e5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:37.984001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a1528-95e3-47ea-a656-323b8c81e5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:37.985955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a1528-95e3-47ea-a656-323b8c81e5b0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:38.006492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1108/2215] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:38.013303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a1528-95e3-47ea-a656-323b8c81e5b0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:38.017742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6751cdb3-7dc8-4d95-86c7-ffb83eb5c802 None None] MFA Rules not processed for user `4dacbeaa7ff34e5bbf95d3ac5f67a049`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:38.032715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6751cdb3-7dc8-4d95-86c7-ffb83eb5c802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:38.037560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a1528-95e3-47ea-a656-323b8c81e5b0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:38.056750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f868a-9e06-4586-9a1d-979b6ff4651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:38.057890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f868a-9e06-4586-9a1d-979b6ff4651d admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:38.058666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f868a-9e06-4586-9a1d-979b6ff4651d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:38.059138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f868a-9e06-4586-9a1d-979b6ff4651d admin admin] PATH_INFO: `/v3/projects/0c899dbcca564df69e3a2b58a3fe50c8/users/5dc028113e864ab7bb24693cad914011/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:38.088942 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f868a-9e06-4586-9a1d-979b6ff4651d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c899dbcca564df69e3a2b58a3fe50c8, user_id=5dc028113e864ab7bb24693cad914011, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:38.092150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f868a-9e06-4586-9a1d-979b6ff4651d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:38.141623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1108/2216] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:38.171994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1109/2217] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:38 2022] PUT /identity/v3/projects/0c899dbcca564df69e3a2b58a3fe50c8/users/5dc028113e864ab7bb24693cad914011/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:38.175908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751cdb3-7dc8-4d95-86c7-ffb83eb5c802 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:38.202705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751cdb3-7dc8-4d95-86c7-ffb83eb5c802 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:38.231707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:38.297887 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:38.350815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b1814-d6af-40e7-afda-ed1103601b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:38.351871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b1814-d6af-40e7-afda-ed1103601b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:38.352683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b1814-d6af-40e7-afda-ed1103601b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:38.353231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b1814-d6af-40e7-afda-ed1103601b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:38.354167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b1814-d6af-40e7-afda-ed1103601b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:38.356997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b1814-d6af-40e7-afda-ed1103601b43 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:38.375151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1109/2218] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:38.380176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f868a-9e06-4586-9a1d-979b6ff4651d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:38.393178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f868a-9e06-4586-9a1d-979b6ff4651d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:38.399556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d37a0f-867a-4b35-9765-fad1ba346b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:38.405989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d37a0f-867a-4b35-9765-fad1ba346b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:38.405989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d37a0f-867a-4b35-9765-fad1ba346b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:38.405989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d37a0f-867a-4b35-9765-fad1ba346b93 admin admin] PATH_INFO: `/v3/projects/0c899dbcca564df69e3a2b58a3fe50c8/users/5dc028113e864ab7bb24693cad914011/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:38.405989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d37a0f-867a-4b35-9765-fad1ba346b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c899dbcca564df69e3a2b58a3fe50c8, user_id=5dc028113e864ab7bb24693cad914011, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:38.407622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d37a0f-867a-4b35-9765-fad1ba346b93 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:38.430343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1110/2219] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:38 2022] PUT /identity/v3/projects/0c899dbcca564df69e3a2b58a3fe50c8/users/5dc028113e864ab7bb24693cad914011/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:38.436832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b1814-d6af-40e7-afda-ed1103601b43 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:38.506204 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b1814-d6af-40e7-afda-ed1103601b43 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:38.577668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b997ce2-c078-4bbc-a27a-bea0cafd2118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:38.578806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b997ce2-c078-4bbc-a27a-bea0cafd2118 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:38.578806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b997ce2-c078-4bbc-a27a-bea0cafd2118 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:38.579231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b997ce2-c078-4bbc-a27a-bea0cafd2118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:38.677165 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b997ce2-c078-4bbc-a27a-bea0cafd2118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:38.678647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b997ce2-c078-4bbc-a27a-bea0cafd2118 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:38.690788 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1110/2220] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:38.717837 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d37a0f-867a-4b35-9765-fad1ba346b93 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:38.808851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d37a0f-867a-4b35-9765-fad1ba346b93 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:38.814468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ffe19-03d1-4a36-9187-b57c9ec85dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:38.815510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ffe19-03d1-4a36-9187-b57c9ec85dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:38.815811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ffe19-03d1-4a36-9187-b57c9ec85dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:38.816147 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ffe19-03d1-4a36-9187-b57c9ec85dbf admin admin] PATH_INFO: `/v3/users/4dacbeaa7ff34e5bbf95d3ac5f67a049` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:38.817393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ffe19-03d1-4a36-9187-b57c9ec85dbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dacbeaa7ff34e5bbf95d3ac5f67a049)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:38.820040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ffe19-03d1-4a36-9187-b57c9ec85dbf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:39.072273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2c7ffe19-03d1-4a36-9187-b57c9ec85dbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dacbeaa7ff34e5bbf95d3ac5f67a049'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:39.095097 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2c7ffe19-03d1-4a36-9187-b57c9ec85dbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dacbeaa7ff34e5bbf95d3ac5f67a049'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:39.107537 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2c7ffe19-03d1-4a36-9187-b57c9ec85dbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dacbeaa7ff34e5bbf95d3ac5f67a049'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:39.124305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b997ce2-c078-4bbc-a27a-bea0cafd2118 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:39.126744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4b997ce2-c078-4bbc-a27a-bea0cafd2118 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:39.152384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b997ce2-c078-4bbc-a27a-bea0cafd2118 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:39.158640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:39.160084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:39.160684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:39.161878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] PATH_INFO: `/v3/users/e470e77147ce4f088750ed66a285d697` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:39.164298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e470e77147ce4f088750ed66a285d697)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:39.167945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:39.191691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2c7ffe19-03d1-4a36-9187-b57c9ec85dbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dacbeaa7ff34e5bbf95d3ac5f67a049'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:39.192311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2c7ffe19-03d1-4a36-9187-b57c9ec85dbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dacbeaa7ff34e5bbf95d3ac5f67a049'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:39.200094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1111/2221] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:38 2022] DELETE /identity/v3/users/4dacbeaa7ff34e5bbf95d3ac5f67a049 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:39.204921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56be3-3c0e-4f5d-9893-9e34ce14ccac None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:39.205540 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56be3-3c0e-4f5d-9893-9e34ce14ccac None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:39.205829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc56be3-3c0e-4f5d-9893-9e34ce14ccac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:39.226567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8cc56be3-3c0e-4f5d-9893-9e34ce14ccac None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:39.227839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8cc56be3-3c0e-4f5d-9893-9e34ce14ccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:39.314112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1112/2222] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:39.338362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc56be3-3c0e-4f5d-9893-9e34ce14ccac None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:39.342737 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8cc56be3-3c0e-4f5d-9893-9e34ce14ccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:39.439339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc56be3-3c0e-4f5d-9893-9e34ce14ccac None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:39.449352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35563d7a-d396-4204-893d-d73c71f2ae9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:39.450849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35563d7a-d396-4204-893d-d73c71f2ae9e service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:39.451384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35563d7a-d396-4204-893d-d73c71f2ae9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:39.451816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35563d7a-d396-4204-893d-d73c71f2ae9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:39.467520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35563d7a-d396-4204-893d-d73c71f2ae9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:39.468254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e470e77147ce4f088750ed66a285d697'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:39.469626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35563d7a-d396-4204-893d-d73c71f2ae9e service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:39.505104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1113/2223] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:39.509328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e470e77147ce4f088750ed66a285d697'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:39.550453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e470e77147ce4f088750ed66a285d697'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:39.574433 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:39.646121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e470e77147ce4f088750ed66a285d697'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:39.647166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e470e77147ce4f088750ed66a285d697'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:39.662729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1111/2224] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:39 2022] DELETE /identity/v3/users/e470e77147ce4f088750ed66a285d697 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:39.667455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a280d-4617-470a-9de8-56d5705ae5dc None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:39.668159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a280d-4617-470a-9de8-56d5705ae5dc None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:39.668534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a280d-4617-470a-9de8-56d5705ae5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:39.690847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-900a280d-4617-470a-9de8-56d5705ae5dc None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:39.692120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-900a280d-4617-470a-9de8-56d5705ae5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:39.825378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1114/2225] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:39.870963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:39.872519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:39.978340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb577c69-720e-46da-8b17-25ac73a827c4 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:39.983243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35338977-8a0e-482c-b716-f09aa49bf272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:39.983911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35338977-8a0e-482c-b716-f09aa49bf272 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:39.989240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35338977-8a0e-482c-b716-f09aa49bf272 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:39.989887 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35338977-8a0e-482c-b716-f09aa49bf272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:40.003026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35338977-8a0e-482c-b716-f09aa49bf272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:40.005714 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35338977-8a0e-482c-b716-f09aa49bf272 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:40.025675 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1112/2226] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:40.118227 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:40.223806 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a280d-4617-470a-9de8-56d5705ae5dc None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:40.226351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-900a280d-4617-470a-9de8-56d5705ae5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:40.250920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a280d-4617-470a-9de8-56d5705ae5dc None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:40.257171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:40.258063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:40.258351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:40.258631 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] PATH_INFO: `/v3/projects/fd163444592a491cb81ee91021815fcf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:40.265843 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd163444592a491cb81ee91021815fcf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:40.269929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:40.476755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd163444592a491cb81ee91021815fcf', 'request_id': 'req-88ca510b-3773-4f77-97c2-efe2017ff6b4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:40.557455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd163444592a491cb81ee91021815fcf', 'request_id': 'req-88ca510b-3773-4f77-97c2-efe2017ff6b4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:40.570916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd163444592a491cb81ee91021815fcf', 'request_id': 'req-88ca510b-3773-4f77-97c2-efe2017ff6b4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:40.596210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] The token cache is being invalidate because project fd163444592a491cb81ee91021815fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:40.596210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:40.600199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-88ca510b-3773-4f77-97c2-efe2017ff6b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:40.604969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1115/2227] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:40 2022] DELETE /identity/v3/projects/fd163444592a491cb81ee91021815fcf => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:40.610337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a030b2-3bed-40d5-9d82-c9af7569d89e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:40.610337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a030b2-3bed-40d5-9d82-c9af7569d89e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:40.610337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a030b2-3bed-40d5-9d82-c9af7569d89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:40.640509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-39a030b2-3bed-40d5-9d82-c9af7569d89e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:40.641811 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-39a030b2-3bed-40d5-9d82-c9af7569d89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:40.836472 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1113/2228] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:40.845248 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60dd4-0f69-49b1-b8cf-d280c26cd690 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:40.845645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60dd4-0f69-49b1-b8cf-d280c26cd690 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:40.845928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f60dd4-0f69-49b1-b8cf-d280c26cd690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:40.868485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:40.881671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-71f60dd4-0f69-49b1-b8cf-d280c26cd690 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:40.883641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-71f60dd4-0f69-49b1-b8cf-d280c26cd690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:40.900239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1116/2229] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:40.915381 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a030b2-3bed-40d5-9d82-c9af7569d89e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:40.959288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a030b2-3bed-40d5-9d82-c9af7569d89e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:40.982111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29b8e8-e9dd-401c-b238-645df1c1da28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:40.986575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29b8e8-e9dd-401c-b238-645df1c1da28 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:40.986921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29b8e8-e9dd-401c-b238-645df1c1da28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:40.987215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29b8e8-e9dd-401c-b238-645df1c1da28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:40.988068 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29b8e8-e9dd-401c-b238-645df1c1da28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:40.991284 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29b8e8-e9dd-401c-b238-645df1c1da28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:40.993121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29b8e8-e9dd-401c-b238-645df1c1da28 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:41.012848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1114/2230] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:41.017412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f60dd4-0f69-49b1-b8cf-d280c26cd690 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:41.032114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f60dd4-0f69-49b1-b8cf-d280c26cd690 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:41.046878 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d01164-ceba-4646-b7f6-11500814d957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:41.048653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d01164-ceba-4646-b7f6-11500814d957 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:41.051968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d01164-ceba-4646-b7f6-11500814d957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:41.052354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d01164-ceba-4646-b7f6-11500814d957 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:41.053316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d01164-ceba-4646-b7f6-11500814d957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:41.057741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d01164-ceba-4646-b7f6-11500814d957 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:41.203262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29b8e8-e9dd-401c-b238-645df1c1da28 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:41.208292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5b29b8e8-e9dd-401c-b238-645df1c1da28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:41.233028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1117/2231] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:41 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:41.241883 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d01164-ceba-4646-b7f6-11500814d957 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:41.260762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29b8e8-e9dd-401c-b238-645df1c1da28 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:41.403405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:41.404323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:41.404663 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:41.404984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] PATH_INFO: `/v3/projects/556c90cae9164495b9d370bc8cd85133` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:41.407143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] RBAC: Authorizing `identity:delete_project(project_id=556c90cae9164495b9d370bc8cd85133)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:41.410064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:41.417979 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:41.455570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d01164-ceba-4646-b7f6-11500814d957 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:41.462760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c9155-838b-4e81-ba75-cc931dd44f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:41.463946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c9155-838b-4e81-ba75-cc931dd44f3c admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:41.464426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c9155-838b-4e81-ba75-cc931dd44f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:41.465326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c9155-838b-4e81-ba75-cc931dd44f3c admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:41.466187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c9155-838b-4e81-ba75-cc931dd44f3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:41.469281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c9155-838b-4e81-ba75-cc931dd44f3c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:41.524105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:41.556637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1118/2232] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:41 2022] POST /identity/v3/users => generated 287 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:41.560297 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c9155-838b-4e81-ba75-cc931dd44f3c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:41.582495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c9155-838b-4e81-ba75-cc931dd44f3c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:41.598411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11478142-1d91-492d-a0a7-1951e0ff510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:41.599191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11478142-1d91-492d-a0a7-1951e0ff510b admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:41.599529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11478142-1d91-492d-a0a7-1951e0ff510b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:41.599812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11478142-1d91-492d-a0a7-1951e0ff510b admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:41.600576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11478142-1d91-492d-a0a7-1951e0ff510b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:41.604346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11478142-1d91-492d-a0a7-1951e0ff510b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:41.629936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1119/2233] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:41.639023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11478142-1d91-492d-a0a7-1951e0ff510b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:41.641963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '556c90cae9164495b9d370bc8cd85133', 'request_id': 'req-2fc6056f-c576-4db1-ba07-4f369975630d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:41.651968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11478142-1d91-492d-a0a7-1951e0ff510b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:41.668109 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '556c90cae9164495b9d370bc8cd85133', 'request_id': 'req-2fc6056f-c576-4db1-ba07-4f369975630d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:41.696525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '556c90cae9164495b9d370bc8cd85133', 'request_id': 'req-2fc6056f-c576-4db1-ba07-4f369975630d'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:41.765539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9a1fb-0383-4cbb-969c-cefd18edf6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:41.766230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9a1fb-0383-4cbb-969c-cefd18edf6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:41.766512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9a1fb-0383-4cbb-969c-cefd18edf6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:41.766860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9a1fb-0383-4cbb-969c-cefd18edf6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:41.767527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9a1fb-0383-4cbb-969c-cefd18edf6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:41.771165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9a1fb-0383-4cbb-969c-cefd18edf6a5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:41.786050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] The token cache is being invalidate because project 556c90cae9164495b9d370bc8cd85133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:41.786307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:41.789653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:41.790834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1115/2234] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:41 2022] DELETE /identity/v3/projects/556c90cae9164495b9d370bc8cd85133 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:41.795449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:41.800740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:41.818592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1120/2235] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:41.818592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9a1fb-0383-4cbb-969c-cefd18edf6a5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:41.832151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cac9a1fb-0383-4cbb-969c-cefd18edf6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:41.835092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6056f-c576-4db1-ba07-4f369975630d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:41.859295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9a1fb-0383-4cbb-969c-cefd18edf6a5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:41.861459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f88692-a780-4e3a-b961-1b0783f62550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:41.862400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f88692-a780-4e3a-b961-1b0783f62550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:41.862680 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f88692-a780-4e3a-b961-1b0783f62550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:41.862951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f88692-a780-4e3a-b961-1b0783f62550 admin admin] PATH_INFO: `/v3/users/3620d0965019471ca0723d82d5bdf3df` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:41.864722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f88692-a780-4e3a-b961-1b0783f62550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3620d0965019471ca0723d82d5bdf3df)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:41.866754 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f88692-a780-4e3a-b961-1b0783f62550 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:41.877040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c13bd-5a96-46f0-867e-082213ecef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:41.878588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c13bd-5a96-46f0-867e-082213ecef45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:41.878879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c13bd-5a96-46f0-867e-082213ecef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:41.879157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c13bd-5a96-46f0-867e-082213ecef45 admin admin] PATH_INFO: `/v3/projects/b4ac8a7f26f6423fb63683053905a4fe/users/cb40c15365c14eb6ae597be773bafc3a/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:41.912378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c13bd-5a96-46f0-867e-082213ecef45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ac8a7f26f6423fb63683053905a4fe, user_id=cb40c15365c14eb6ae597be773bafc3a, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:41.914806 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c13bd-5a96-46f0-867e-082213ecef45 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:42.079601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1121/2236] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:41 2022] PUT /identity/v3/projects/b4ac8a7f26f6423fb63683053905a4fe/users/cb40c15365c14eb6ae597be773bafc3a/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:42.090624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed2140-55fb-48c2-9f98-99da8c45ab4c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:42.090920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed2140-55fb-48c2-9f98-99da8c45ab4c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:42.091196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed2140-55fb-48c2-9f98-99da8c45ab4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:42.144028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5bed2140-55fb-48c2-9f98-99da8c45ab4c None None] MFA Rules not processed for user `cb40c15365c14eb6ae597be773bafc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:42.145701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5bed2140-55fb-48c2-9f98-99da8c45ab4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:42.203621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-34f88692-a780-4e3a-b961-1b0783f62550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3620d0965019471ca0723d82d5bdf3df'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:42.235867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-34f88692-a780-4e3a-b961-1b0783f62550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3620d0965019471ca0723d82d5bdf3df'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:42.297112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-34f88692-a780-4e3a-b961-1b0783f62550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3620d0965019471ca0723d82d5bdf3df'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:42.311568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1122/2237] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:42.336577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed2140-55fb-48c2-9f98-99da8c45ab4c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:42.338426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5bed2140-55fb-48c2-9f98-99da8c45ab4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:42.370208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-34f88692-a780-4e3a-b961-1b0783f62550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3620d0965019471ca0723d82d5bdf3df'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:42.370981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-34f88692-a780-4e3a-b961-1b0783f62550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3620d0965019471ca0723d82d5bdf3df'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:42.378886 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1116/2238] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:41 2022] DELETE /identity/v3/users/3620d0965019471ca0723d82d5bdf3df => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:42.426436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed2140-55fb-48c2-9f98-99da8c45ab4c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:42.557070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bcbf4-7688-434d-9053-b2b76cfff1e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:42.557772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bcbf4-7688-434d-9053-b2b76cfff1e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:42.558064 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bcbf4-7688-434d-9053-b2b76cfff1e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:42.558345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bcbf4-7688-434d-9053-b2b76cfff1e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:42.560441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b65bcbf4-7688-434d-9053-b2b76cfff1e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:42.666116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bcbf4-7688-434d-9053-b2b76cfff1e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:42.668246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bcbf4-7688-434d-9053-b2b76cfff1e6 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:42.706839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:42.717638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1123/2239] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:42.792796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784c3de-51cc-4346-a8fd-38587a23965f None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:42.792796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784c3de-51cc-4346-a8fd-38587a23965f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:42.792796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784c3de-51cc-4346-a8fd-38587a23965f None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:42.799644 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1117/2240] 10.222.0.88 () {54 vars in 921 bytes} [Mon Jul 18 16:51:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:51:42.802320 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f05dc-e04f-4012-a5e8-483d6f5df0f7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:42.802615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f05dc-e04f-4012-a5e8-483d6f5df0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:42.802914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f05dc-e04f-4012-a5e8-483d6f5df0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:42.827927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-212f05dc-e04f-4012-a5e8-483d6f5df0f7 None None] MFA Rules not processed for user `22bd4dab1fcd4ad4a8be886646d89242`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:42.830176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-212f05dc-e04f-4012-a5e8-483d6f5df0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:42.923639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1124/2241] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Jul 18 16:51:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:42.948795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784c3de-51cc-4346-a8fd-38587a23965f None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:42.951007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7784c3de-51cc-4346-a8fd-38587a23965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:42.986145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f05dc-e04f-4012-a5e8-483d6f5df0f7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:42.989759 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-212f05dc-e04f-4012-a5e8-483d6f5df0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:43.003677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784c3de-51cc-4346-a8fd-38587a23965f None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:43.016568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e27910-32a8-43d3-8768-4b1fd85553d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:43.017279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e27910-32a8-43d3-8768-4b1fd85553d9 service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:43.017564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e27910-32a8-43d3-8768-4b1fd85553d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:43.017865 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e27910-32a8-43d3-8768-4b1fd85553d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:43.040086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e27910-32a8-43d3-8768-4b1fd85553d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:43.041834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e27910-32a8-43d3-8768-4b1fd85553d9 service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:43.066871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:43.068708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1118/2242] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:43.108909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f05dc-e04f-4012-a5e8-483d6f5df0f7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:43.118396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9783d6-5b65-4ff1-b7ed-aad60017dc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:43.121276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9783d6-5b65-4ff1-b7ed-aad60017dc84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:43.121276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9783d6-5b65-4ff1-b7ed-aad60017dc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:43.121276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9783d6-5b65-4ff1-b7ed-aad60017dc84 admin admin] PATH_INFO: `/v3/users/55e8b1fa83fb4e9593dc60735feeb01b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:43.121276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9783d6-5b65-4ff1-b7ed-aad60017dc84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e8b1fa83fb4e9593dc60735feeb01b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:43.123325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9783d6-5b65-4ff1-b7ed-aad60017dc84 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:43.435069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cb9783d6-5b65-4ff1-b7ed-aad60017dc84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e8b1fa83fb4e9593dc60735feeb01b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:43.466393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cb9783d6-5b65-4ff1-b7ed-aad60017dc84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e8b1fa83fb4e9593dc60735feeb01b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:43.482169 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e27910-32a8-43d3-8768-4b1fd85553d9 service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:43.484679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a9e27910-32a8-43d3-8768-4b1fd85553d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:43.486285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cb9783d6-5b65-4ff1-b7ed-aad60017dc84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e8b1fa83fb4e9593dc60735feeb01b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:43.504511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e27910-32a8-43d3-8768-4b1fd85553d9 service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:43.518556 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:43.519965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:43.520439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:43.520996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] PATH_INFO: `/v3/projects/aa62f581ec32489ba569f7e7826b4287` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:43.522498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa62f581ec32489ba569f7e7826b4287)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:43.524963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:43.568430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cb9783d6-5b65-4ff1-b7ed-aad60017dc84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e8b1fa83fb4e9593dc60735feeb01b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:43.569161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cb9783d6-5b65-4ff1-b7ed-aad60017dc84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e8b1fa83fb4e9593dc60735feeb01b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:43.576941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1125/2243] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:42 2022] DELETE /identity/v3/users/55e8b1fa83fb4e9593dc60735feeb01b => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:43.582008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640d754-f58e-4c93-9f70-8718b5feb5df None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:43.582423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640d754-f58e-4c93-9f70-8718b5feb5df None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:43.582791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640d754-f58e-4c93-9f70-8718b5feb5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:43.623658 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-7640d754-f58e-4c93-9f70-8718b5feb5df None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:43.626446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7640d754-f58e-4c93-9f70-8718b5feb5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:43.713338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa62f581ec32489ba569f7e7826b4287', 'request_id': 'req-7a661a44-2d9e-4155-bdea-ccff0b1aa979'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:43.726821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa62f581ec32489ba569f7e7826b4287', 'request_id': 'req-7a661a44-2d9e-4155-bdea-ccff0b1aa979'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:43.739590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa62f581ec32489ba569f7e7826b4287', 'request_id': 'req-7a661a44-2d9e-4155-bdea-ccff0b1aa979'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:43.782683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1126/2244] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:43.827423 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] The token cache is being invalidate because project aa62f581ec32489ba569f7e7826b4287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:43.827962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:43.828414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-7a661a44-2d9e-4155-bdea-ccff0b1aa979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:43.832187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640d754-f58e-4c93-9f70-8718b5feb5df None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:43.834290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7640d754-f58e-4c93-9f70-8718b5feb5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:43.841461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1119/2245] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:43 2022] DELETE /identity/v3/projects/aa62f581ec32489ba569f7e7826b4287 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:43.846241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdca883-4eb9-4cc8-b86f-4f1c2dcac2bb None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:43.848945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdca883-4eb9-4cc8-b86f-4f1c2dcac2bb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:43.849692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdca883-4eb9-4cc8-b86f-4f1c2dcac2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:43.902195 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8bdca883-4eb9-4cc8-b86f-4f1c2dcac2bb None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:43.905298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8bdca883-4eb9-4cc8-b86f-4f1c2dcac2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:43.982675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640d754-f58e-4c93-9f70-8718b5feb5df None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:44.037736 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1120/2246] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:44.046001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45174e-d889-4a2a-b33c-3d45c681ffe7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:44.046515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45174e-d889-4a2a-b33c-3d45c681ffe7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:44.047021 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45174e-d889-4a2a-b33c-3d45c681ffe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:44.066874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c3c5-8ad4-4906-b292-e9c8272cf28f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:44.067553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c3c5-8ad4-4906-b292-e9c8272cf28f service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:44.068065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c3c5-8ad4-4906-b292-e9c8272cf28f service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:44.068354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c3c5-8ad4-4906-b292-e9c8272cf28f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:44.072451 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ae39c3c5-8ad4-4906-b292-e9c8272cf28f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:44.088409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39c3c5-8ad4-4906-b292-e9c8272cf28f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:44.090988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39c3c5-8ad4-4906-b292-e9c8272cf28f service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:44.094877 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ba45174e-d889-4a2a-b33c-3d45c681ffe7 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:44.097885 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ba45174e-d889-4a2a-b33c-3d45c681ffe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:44.109767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1121/2247] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:44.114323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45174e-d889-4a2a-b33c-3d45c681ffe7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:44.121291 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:44.127631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45174e-d889-4a2a-b33c-3d45c681ffe7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:44.131163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1127/2248] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:44.137267 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fcad5-06b5-4841-8a59-89f04a1ce18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:44.139262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fcad5-06b5-4841-8a59-89f04a1ce18f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:44.139262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fcad5-06b5-4841-8a59-89f04a1ce18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:44.139262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fcad5-06b5-4841-8a59-89f04a1ce18f admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:44.140195 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fcad5-06b5-4841-8a59-89f04a1ce18f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:44.140674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fcad5-06b5-4841-8a59-89f04a1ce18f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:44.143254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fcad5-06b5-4841-8a59-89f04a1ce18f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:44.161572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1122/2249] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:44.166494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c3c5-8ad4-4906-b292-e9c8272cf28f service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:44.179902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c3c5-8ad4-4906-b292-e9c8272cf28f service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:44.186003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d86af-8aa3-4b75-acfd-5deeb46d475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:44.186924 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d86af-8aa3-4b75-acfd-5deeb46d475e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:44.187458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d86af-8aa3-4b75-acfd-5deeb46d475e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:44.187831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d86af-8aa3-4b75-acfd-5deeb46d475e admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:44.188670 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d86af-8aa3-4b75-acfd-5deeb46d475e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:44.190852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d86af-8aa3-4b75-acfd-5deeb46d475e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:44.221537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fcad5-06b5-4841-8a59-89f04a1ce18f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:44.225405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4f6fcad5-06b5-4841-8a59-89f04a1ce18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:44.242070 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fcad5-06b5-4841-8a59-89f04a1ce18f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:44.247907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a536cf-c8dc-49e7-b945-ced4a8be7fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:44.249274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a536cf-c8dc-49e7-b945-ced4a8be7fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:44.249787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a536cf-c8dc-49e7-b945-ced4a8be7fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:44.250397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a536cf-c8dc-49e7-b945-ced4a8be7fc4 admin admin] PATH_INFO: `/v3/users/8617eb9fff41436d867e205da136d61f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:44.251976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a536cf-c8dc-49e7-b945-ced4a8be7fc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8617eb9fff41436d867e205da136d61f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:44.254389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a536cf-c8dc-49e7-b945-ced4a8be7fc4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:44.283506 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=76,keystone:DELETE=21,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:44.324807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1128/2250] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:44 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:51:44.328916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d86af-8aa3-4b75-acfd-5deeb46d475e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:44.406926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d86af-8aa3-4b75-acfd-5deeb46d475e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:44.406926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a0712-e6e4-471b-a95d-c8a7bd437c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:44.407520 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a0712-e6e4-471b-a95d-c8a7bd437c56 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:44.408009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a0712-e6e4-471b-a95d-c8a7bd437c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:44.408814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a0712-e6e4-471b-a95d-c8a7bd437c56 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:44.410433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a0712-e6e4-471b-a95d-c8a7bd437c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:44.414773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a0712-e6e4-471b-a95d-c8a7bd437c56 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:44.535026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1129/2251] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:44 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:44.538944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a0712-e6e4-471b-a95d-c8a7bd437c56 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:44.545038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-74a536cf-c8dc-49e7-b945-ced4a8be7fc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8617eb9fff41436d867e205da136d61f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:44.551926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a0712-e6e4-471b-a95d-c8a7bd437c56 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:44.555972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-74a536cf-c8dc-49e7-b945-ced4a8be7fc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8617eb9fff41436d867e205da136d61f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:44.557740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f66ec-daa7-4e84-b366-2842ced6a734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:44.559271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f66ec-daa7-4e84-b366-2842ced6a734 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:44.559467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f66ec-daa7-4e84-b366-2842ced6a734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:44.559735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f66ec-daa7-4e84-b366-2842ced6a734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:44.560488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f66ec-daa7-4e84-b366-2842ced6a734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:44.562268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f66ec-daa7-4e84-b366-2842ced6a734 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:44.571490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-74a536cf-c8dc-49e7-b945-ced4a8be7fc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8617eb9fff41436d867e205da136d61f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:44.576928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1130/2252] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:44.579746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f66ec-daa7-4e84-b366-2842ced6a734 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:44.591194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f66ec-daa7-4e84-b366-2842ced6a734 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:44.597763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01ebb2-db77-4dbc-bdbc-45e89e258f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:44.598767 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01ebb2-db77-4dbc-bdbc-45e89e258f97 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:44.599070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01ebb2-db77-4dbc-bdbc-45e89e258f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:44.599337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01ebb2-db77-4dbc-bdbc-45e89e258f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:44.600151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01ebb2-db77-4dbc-bdbc-45e89e258f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:44.601756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01ebb2-db77-4dbc-bdbc-45e89e258f97 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:44.617052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1131/2253] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:44.618466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01ebb2-db77-4dbc-bdbc-45e89e258f97 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:44.629365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01ebb2-db77-4dbc-bdbc-45e89e258f97 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:44.634495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7582e-961f-406d-a6f3-0338c79bb683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:44.635390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7582e-961f-406d-a6f3-0338c79bb683 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:44.635770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7582e-961f-406d-a6f3-0338c79bb683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:44.636529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7582e-961f-406d-a6f3-0338c79bb683 admin admin] PATH_INFO: `/v3/projects/a2547613e85441299690cab07f6b23f2/users/1625659dc8294a7583fc566d8397bbed/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:44.655114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7582e-961f-406d-a6f3-0338c79bb683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2547613e85441299690cab07f6b23f2, user_id=1625659dc8294a7583fc566d8397bbed, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:44.658164 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7582e-961f-406d-a6f3-0338c79bb683 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:44.682896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-74a536cf-c8dc-49e7-b945-ced4a8be7fc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8617eb9fff41436d867e205da136d61f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:44.683531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-74a536cf-c8dc-49e7-b945-ced4a8be7fc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8617eb9fff41436d867e205da136d61f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:44.691487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1123/2254] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:44 2022] DELETE /identity/v3/users/8617eb9fff41436d867e205da136d61f => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:44.808032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1132/2255] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:44 2022] PUT /identity/v3/projects/a2547613e85441299690cab07f6b23f2/users/1625659dc8294a7583fc566d8397bbed/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:44.820741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d074e68-b98b-4bf4-9b9c-d71e617a909b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:44.820741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d074e68-b98b-4bf4-9b9c-d71e617a909b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:44.821099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d074e68-b98b-4bf4-9b9c-d71e617a909b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:44.884354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d0307-c779-4ce6-8dee-f7e1ded81248 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:44.884998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d0307-c779-4ce6-8dee-f7e1ded81248 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:44.885560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d0307-c779-4ce6-8dee-f7e1ded81248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:44.898136 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6d074e68-b98b-4bf4-9b9c-d71e617a909b None None] MFA Rules not processed for user `1625659dc8294a7583fc566d8397bbed`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:44.900372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6d074e68-b98b-4bf4-9b9c-d71e617a909b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:44.922753 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-221d0307-c779-4ce6-8dee-f7e1ded81248 None None] MFA Rules not processed for user `cb40c15365c14eb6ae597be773bafc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:44.923156 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-221d0307-c779-4ce6-8dee-f7e1ded81248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:45.021882 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1124/2256] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:45.023042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d074e68-b98b-4bf4-9b9c-d71e617a909b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:45.029669 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1133/2257] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:45.033483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d0307-c779-4ce6-8dee-f7e1ded81248 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:45.038476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6d074e68-b98b-4bf4-9b9c-d71e617a909b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:45.038941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-221d0307-c779-4ce6-8dee-f7e1ded81248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:45.139440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d0307-c779-4ce6-8dee-f7e1ded81248 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:45.149399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2b989-4f35-42c8-9142-1867e76b2ad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:45.150245 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2b989-4f35-42c8-9142-1867e76b2ad6 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:45.150599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2b989-4f35-42c8-9142-1867e76b2ad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:45.150877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2b989-4f35-42c8-9142-1867e76b2ad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:45.171979 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2b989-4f35-42c8-9142-1867e76b2ad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:45.176515 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2b989-4f35-42c8-9142-1867e76b2ad6 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:45.207291 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d074e68-b98b-4bf4-9b9c-d71e617a909b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:45.220535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1134/2258] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:45.224622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2b989-4f35-42c8-9142-1867e76b2ad6 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:45.226122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-86b2b989-4f35-42c8-9142-1867e76b2ad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:45.227292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:45.230611 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:45.230910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:45.231191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] PATH_INFO: `/v3/projects/2fcd784fd30e41629dd3449eb4969e34` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:45.236925 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fcd784fd30e41629dd3449eb4969e34)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:45.241158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:45.269941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2b989-4f35-42c8-9142-1867e76b2ad6 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:45.293659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:45.300267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:45.300267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:45.300267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] PATH_INFO: `/v3/projects/2a8580d1c8784e78aee4f19aa83d0823` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:45.300267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8580d1c8784e78aee4f19aa83d0823)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:45.303968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:45.449410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fcd784fd30e41629dd3449eb4969e34', 'request_id': 'req-8449c1c8-9399-48dd-b8f6-991e72ef6018'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.475674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fcd784fd30e41629dd3449eb4969e34', 'request_id': 'req-8449c1c8-9399-48dd-b8f6-991e72ef6018'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.491348 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8580d1c8784e78aee4f19aa83d0823', 'request_id': 'req-81fbd132-4673-4ab2-9311-42690275d1c2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.503582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fcd784fd30e41629dd3449eb4969e34', 'request_id': 'req-8449c1c8-9399-48dd-b8f6-991e72ef6018'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.580364 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8580d1c8784e78aee4f19aa83d0823', 'request_id': 'req-81fbd132-4673-4ab2-9311-42690275d1c2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.588124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8580d1c8784e78aee4f19aa83d0823', 'request_id': 'req-81fbd132-4673-4ab2-9311-42690275d1c2'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.615106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] The token cache is being invalidate because project 2a8580d1c8784e78aee4f19aa83d0823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:45.615408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.624829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.624829 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1135/2259] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:45 2022] DELETE /identity/v3/projects/2a8580d1c8784e78aee4f19aa83d0823 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:45.625250 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:45.631380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:45.679193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] The token cache is being invalidate because project 2fcd784fd30e41629dd3449eb4969e34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:45.679484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.680622 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.684751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1125/2260] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:45 2022] DELETE /identity/v3/projects/2fcd784fd30e41629dd3449eb4969e34 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:45.688553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:45.690671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:45.752052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbd132-4673-4ab2-9311-42690275d1c2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:45.761362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:45.762476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:45.762768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:45.763055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] PATH_INFO: `/v3/users/d572114f5f234aa0a399494644d47254` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:45.765748 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d572114f5f234aa0a399494644d47254)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:45.769978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:45.812870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449c1c8-9399-48dd-b8f6-991e72ef6018 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:45.842347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cafec6-5e49-4924-8791-86b449a827b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:45.843247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cafec6-5e49-4924-8791-86b449a827b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:45.843529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cafec6-5e49-4924-8791-86b449a827b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:45.843804 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cafec6-5e49-4924-8791-86b449a827b1 admin admin] PATH_INFO: `/v3/users/7ba145b5874c4f2ca3d9e4ec1c328181` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:45.846338 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cafec6-5e49-4924-8791-86b449a827b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba145b5874c4f2ca3d9e4ec1c328181)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:45.849056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cafec6-5e49-4924-8791-86b449a827b1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:45.952015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:45.954997 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd572114f5f234aa0a399494644d47254'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.982628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd572114f5f234aa0a399494644d47254'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:45.997050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd572114f5f234aa0a399494644d47254'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:46.077465 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b6cafec6-5e49-4924-8791-86b449a827b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba145b5874c4f2ca3d9e4ec1c328181'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:46.083930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd572114f5f234aa0a399494644d47254'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:46.085213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd572114f5f234aa0a399494644d47254'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:46.101085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1136/2261] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:45 2022] DELETE /identity/v3/users/d572114f5f234aa0a399494644d47254 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:46.106139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b6cafec6-5e49-4924-8791-86b449a827b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba145b5874c4f2ca3d9e4ec1c328181'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:46.121743 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b6cafec6-5e49-4924-8791-86b449a827b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba145b5874c4f2ca3d9e4ec1c328181'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:46.176207 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:46.198646 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b6cafec6-5e49-4924-8791-86b449a827b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba145b5874c4f2ca3d9e4ec1c328181'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:46.201633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b6cafec6-5e49-4924-8791-86b449a827b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba145b5874c4f2ca3d9e4ec1c328181'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:46.226301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1126/2262] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:45 2022] DELETE /identity/v3/users/7ba145b5874c4f2ca3d9e4ec1c328181 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:46.835473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:46.838950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:47.053251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939495bd-6af4-4512-ad32-8fbe641fcab9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:47.187055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:47.188088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:47.188470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:47.189428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] PATH_INFO: `/v3/projects/a6cce398ee6f4f5183e7f24c3da5f70b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:47.191501 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6cce398ee6f4f5183e7f24c3da5f70b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:47.193711 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:47.383176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6cce398ee6f4f5183e7f24c3da5f70b', 'request_id': 'req-b6eae4ff-1365-4de2-81d5-0fba95723b3d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:47.469743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:47.471828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6cce398ee6f4f5183e7f24c3da5f70b', 'request_id': 'req-b6eae4ff-1365-4de2-81d5-0fba95723b3d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:47.492450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6cce398ee6f4f5183e7f24c3da5f70b', 'request_id': 'req-b6eae4ff-1365-4de2-81d5-0fba95723b3d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:47.553341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] The token cache is being invalidate because project a6cce398ee6f4f5183e7f24c3da5f70b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:47.553719 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:47.554492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b6eae4ff-1365-4de2-81d5-0fba95723b3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:47.556208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1137/2263] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:46 2022] DELETE /identity/v3/projects/a6cce398ee6f4f5183e7f24c3da5f70b => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:47.562331 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d37e2-07d1-46e5-8faa-2585315edd07 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:47.563072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d37e2-07d1-46e5-8faa-2585315edd07 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:47.563072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d37e2-07d1-46e5-8faa-2585315edd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:47.604456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-120d37e2-07d1-46e5-8faa-2585315edd07 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:47.608190 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-120d37e2-07d1-46e5-8faa-2585315edd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:47.720105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1127/2264] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:47.728708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dc1b2-9584-4ec5-a8de-48e88c36e722 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:47.729042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dc1b2-9584-4ec5-a8de-48e88c36e722 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:47.733890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534dc1b2-9584-4ec5-a8de-48e88c36e722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:47.764442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d37e2-07d1-46e5-8faa-2585315edd07 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:47.764442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-120d37e2-07d1-46e5-8faa-2585315edd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:47.776272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-534dc1b2-9584-4ec5-a8de-48e88c36e722 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:47.777496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-534dc1b2-9584-4ec5-a8de-48e88c36e722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:47.795902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d37e2-07d1-46e5-8faa-2585315edd07 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:47.816446 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:47.817831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:47.818492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:47.818931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] PATH_INFO: `/v3/projects/05dbb1af9525455eb1fc6888e965de49` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:47.821187 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05dbb1af9525455eb1fc6888e965de49)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:47.823194 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:47.828004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1138/2265] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:47.831850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dc1b2-9584-4ec5-a8de-48e88c36e722 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:47.845036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534dc1b2-9584-4ec5-a8de-48e88c36e722 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:47.854332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164aa444-9860-44b9-b2f8-81d99798e265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:47.855027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164aa444-9860-44b9-b2f8-81d99798e265 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:47.855311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164aa444-9860-44b9-b2f8-81d99798e265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:47.855585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164aa444-9860-44b9-b2f8-81d99798e265 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:47.856348 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164aa444-9860-44b9-b2f8-81d99798e265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:47.856645 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164aa444-9860-44b9-b2f8-81d99798e265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:47.858353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164aa444-9860-44b9-b2f8-81d99798e265 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:47.885680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1139/2266] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:47.890197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164aa444-9860-44b9-b2f8-81d99798e265 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:47.902909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164aa444-9860-44b9-b2f8-81d99798e265 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:47.918414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa82e7-b1a7-45b0-ba3c-c424fa256640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:47.919329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa82e7-b1a7-45b0-ba3c-c424fa256640 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:47.919908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa82e7-b1a7-45b0-ba3c-c424fa256640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:47.920867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa82e7-b1a7-45b0-ba3c-c424fa256640 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:47.921770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa82e7-b1a7-45b0-ba3c-c424fa256640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:47.923863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa82e7-b1a7-45b0-ba3c-c424fa256640 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:48.068087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1140/2267] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:47 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:48.072455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa82e7-b1a7-45b0-ba3c-c424fa256640 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:48.075089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05dbb1af9525455eb1fc6888e965de49', 'request_id': 'req-25ce2009-530e-45d3-aad2-4676a1342882'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:48.098783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05dbb1af9525455eb1fc6888e965de49', 'request_id': 'req-25ce2009-530e-45d3-aad2-4676a1342882'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:48.123556 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05dbb1af9525455eb1fc6888e965de49', 'request_id': 'req-25ce2009-530e-45d3-aad2-4676a1342882'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:48.226197 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] The token cache is being invalidate because project 05dbb1af9525455eb1fc6888e965de49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:48.226677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:48.227774 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:48.247109 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1128/2268] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:47 2022] DELETE /identity/v3/projects/05dbb1af9525455eb1fc6888e965de49 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:48.254541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:48.264285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:48.302066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:48.364178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa82e7-b1a7-45b0-ba3c-c424fa256640 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:48.375238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4001b92-fb87-4a00-b232-4898f2ee84b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:48.376703 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4001b92-fb87-4a00-b232-4898f2ee84b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:48.377659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4001b92-fb87-4a00-b232-4898f2ee84b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:48.378342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4001b92-fb87-4a00-b232-4898f2ee84b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:48.381324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4001b92-fb87-4a00-b232-4898f2ee84b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:48.385617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4001b92-fb87-4a00-b232-4898f2ee84b1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:48.462770 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce2009-530e-45d3-aad2-4676a1342882 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:48.474309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:48.475264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:48.475557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:48.476198 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] PATH_INFO: `/v3/users/36450aa5e6e94c4f97b75e60fc726724` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:48.484297 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:48.489841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] RBAC: Authorizing `identity:delete_user(user_id=36450aa5e6e94c4f97b75e60fc726724)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:48.497118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1141/2269] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:48 2022] POST /identity/v3/users => generated 289 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:48.501544 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4001b92-fb87-4a00-b232-4898f2ee84b1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:48.503643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a4001b92-fb87-4a00-b232-4898f2ee84b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:48.505685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:48.528288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4001b92-fb87-4a00-b232-4898f2ee84b1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:48.541395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa6374-33b9-4666-9a3e-f49fc5a0e32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:48.542158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa6374-33b9-4666-9a3e-f49fc5a0e32e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:48.542568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa6374-33b9-4666-9a3e-f49fc5a0e32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:48.543025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa6374-33b9-4666-9a3e-f49fc5a0e32e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:48.543988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa6374-33b9-4666-9a3e-f49fc5a0e32e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:48.545855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa6374-33b9-4666-9a3e-f49fc5a0e32e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:48.571903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1142/2270] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:51:48.575663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa6374-33b9-4666-9a3e-f49fc5a0e32e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:48.595537 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa6374-33b9-4666-9a3e-f49fc5a0e32e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:48.604031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc61a3-e269-4a28-b154-4645f9de4a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:48.604252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc61a3-e269-4a28-b154-4645f9de4a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:48.604634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc61a3-e269-4a28-b154-4645f9de4a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:48.605001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc61a3-e269-4a28-b154-4645f9de4a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:48.605751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc61a3-e269-4a28-b154-4645f9de4a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:48.607458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc61a3-e269-4a28-b154-4645f9de4a93 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:48.620459 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1143/2271] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:48.624631 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc61a3-e269-4a28-b154-4645f9de4a93 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:48.635272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc61a3-e269-4a28-b154-4645f9de4a93 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:48.640871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534ed6a-4933-4200-aabf-5b372177cdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:48.641838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534ed6a-4933-4200-aabf-5b372177cdcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:48.642118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534ed6a-4933-4200-aabf-5b372177cdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:48.642445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534ed6a-4933-4200-aabf-5b372177cdcc admin admin] PATH_INFO: `/v3/projects/a2edfc0ea1824972bc4ceacb03da421c/users/f14600f242a8490ca4d83b6c6ad2ab47/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:48.672969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534ed6a-4933-4200-aabf-5b372177cdcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2edfc0ea1824972bc4ceacb03da421c, user_id=f14600f242a8490ca4d83b6c6ad2ab47, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:48.676485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534ed6a-4933-4200-aabf-5b372177cdcc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:48.791939 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1144/2272] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:48 2022] PUT /identity/v3/projects/a2edfc0ea1824972bc4ceacb03da421c/users/f14600f242a8490ca4d83b6c6ad2ab47/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:48.803625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135f9ff-63ab-4bee-bfc4-f7c3ffadc72e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:48.807406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135f9ff-63ab-4bee-bfc4-f7c3ffadc72e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:48.807406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135f9ff-63ab-4bee-bfc4-f7c3ffadc72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:48.807596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36450aa5e6e94c4f97b75e60fc726724'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:48.836472 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6135f9ff-63ab-4bee-bfc4-f7c3ffadc72e None None] MFA Rules not processed for user `f14600f242a8490ca4d83b6c6ad2ab47`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:48.837438 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6135f9ff-63ab-4bee-bfc4-f7c3ffadc72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:48.843328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36450aa5e6e94c4f97b75e60fc726724'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:48.872095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36450aa5e6e94c4f97b75e60fc726724'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:48.940062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1145/2273] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:48.952133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135f9ff-63ab-4bee-bfc4-f7c3ffadc72e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:48.953887 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6135f9ff-63ab-4bee-bfc4-f7c3ffadc72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:48.974875 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36450aa5e6e94c4f97b75e60fc726724'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:48.975698 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36450aa5e6e94c4f97b75e60fc726724'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:48.986981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1129/2274] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:48 2022] DELETE /identity/v3/users/36450aa5e6e94c4f97b75e60fc726724 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:49.070614 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135f9ff-63ab-4bee-bfc4-f7c3ffadc72e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:49.161919 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62e129-27e0-4622-b6e2-55833f0ecce9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54c7869983bf4534b3e730af0185eac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:49.162810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62e129-27e0-4622-b6e2-55833f0ecce9 service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:49.162932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62e129-27e0-4622-b6e2-55833f0ecce9 service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:49.163179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62e129-27e0-4622-b6e2-55833f0ecce9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:49.165311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1f62e129-27e0-4622-b6e2-55833f0ecce9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:49.246692 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62e129-27e0-4622-b6e2-55833f0ecce9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:49.246692 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62e129-27e0-4622-b6e2-55833f0ecce9 service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:49.262896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1146/2275] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:49.341436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:49.558906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:49.558906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:49.588898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:49.684330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563a4bd-edd8-409f-8811-1f89eea02bee admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:49.695862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:49.695862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:49.695862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:49.695862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] PATH_INFO: `/v3/projects/6aaca002713b437b85a40ddeb8351c4f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:49.696245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aaca002713b437b85a40ddeb8351c4f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:49.697809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:49.922618 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aaca002713b437b85a40ddeb8351c4f', 'request_id': 'req-9d81cb75-0a25-46d6-b584-403404a73938'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:49.935028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aaca002713b437b85a40ddeb8351c4f', 'request_id': 'req-9d81cb75-0a25-46d6-b584-403404a73938'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:49.949399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aaca002713b437b85a40ddeb8351c4f', 'request_id': 'req-9d81cb75-0a25-46d6-b584-403404a73938'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:50.032900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] The token cache is being invalidate because project 6aaca002713b437b85a40ddeb8351c4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:50.037221 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:50.037221 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9d81cb75-0a25-46d6-b584-403404a73938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:50.037608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1130/2276] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:49 2022] DELETE /identity/v3/projects/6aaca002713b437b85a40ddeb8351c4f => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:50.037858 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:50.043853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30871b-cd06-450d-96a9-3bbea011217a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:50.044152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30871b-cd06-450d-96a9-3bbea011217a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:50.044436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30871b-cd06-450d-96a9-3bbea011217a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:50.091352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8b30871b-cd06-450d-96a9-3bbea011217a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:50.092969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8b30871b-cd06-450d-96a9-3bbea011217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:50.164383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8faeb-721d-4bec-acb1-a58f7bb774d1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:50.165116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8faeb-721d-4bec-acb1-a58f7bb774d1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:50.167569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8faeb-721d-4bec-acb1-a58f7bb774d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:50.215617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1147/2277] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:50.225678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d742b7-43b8-41de-b5c7-e09ea8e0f35a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:50.225904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d742b7-43b8-41de-b5c7-e09ea8e0f35a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:50.226153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d742b7-43b8-41de-b5c7-e09ea8e0f35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:50.232106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-04f8faeb-721d-4bec-acb1-a58f7bb774d1 None None] MFA Rules not processed for user `5dc028113e864ab7bb24693cad914011`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:50.237866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-04f8faeb-721d-4bec-acb1-a58f7bb774d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:50.251513 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-35d742b7-43b8-41de-b5c7-e09ea8e0f35a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:50.253345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-35d742b7-43b8-41de-b5c7-e09ea8e0f35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:50.265566 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1148/2278] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:51:50.266903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d742b7-43b8-41de-b5c7-e09ea8e0f35a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:50.278974 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d742b7-43b8-41de-b5c7-e09ea8e0f35a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:50.286581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200a8bd-2a5e-4d33-91e6-7641069bab33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:50.287341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200a8bd-2a5e-4d33-91e6-7641069bab33 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:50.287703 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200a8bd-2a5e-4d33-91e6-7641069bab33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:50.287984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200a8bd-2a5e-4d33-91e6-7641069bab33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:50.289621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200a8bd-2a5e-4d33-91e6-7641069bab33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:50.289946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200a8bd-2a5e-4d33-91e6-7641069bab33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:50.293991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200a8bd-2a5e-4d33-91e6-7641069bab33 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:50.319873 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1149/2279] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:50.320060 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200a8bd-2a5e-4d33-91e6-7641069bab33 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:50.332772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200a8bd-2a5e-4d33-91e6-7641069bab33 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:50.345043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b416d-27ac-401b-aacc-769cb27b891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:50.345043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b416d-27ac-401b-aacc-769cb27b891f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:50.345043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b416d-27ac-401b-aacc-769cb27b891f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:50.345043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b416d-27ac-401b-aacc-769cb27b891f admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:50.345524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b416d-27ac-401b-aacc-769cb27b891f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:50.345524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b416d-27ac-401b-aacc-769cb27b891f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:50.437056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1131/2280] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:50.454703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8faeb-721d-4bec-acb1-a58f7bb774d1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:50.456393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-04f8faeb-721d-4bec-acb1-a58f7bb774d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:50.499796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1150/2281] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:50 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:50.501733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b416d-27ac-401b-aacc-769cb27b891f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:50.583027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8faeb-721d-4bec-acb1-a58f7bb774d1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:50.647383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b416d-27ac-401b-aacc-769cb27b891f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:50.652910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71c290-e4be-4674-a56a-ecf4c619f70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:50.653614 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c290-e4be-4674-a56a-ecf4c619f70c admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:50.653901 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c290-e4be-4674-a56a-ecf4c619f70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:50.654177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c290-e4be-4674-a56a-ecf4c619f70c admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:50.654935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71c290-e4be-4674-a56a-ecf4c619f70c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:50.658262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03645078-8fdb-4d3f-b3e3-3c4a06a05317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:50.659644 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645078-8fdb-4d3f-b3e3-3c4a06a05317 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:50.659997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645078-8fdb-4d3f-b3e3-3c4a06a05317 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:50.660277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645078-8fdb-4d3f-b3e3-3c4a06a05317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:50.662039 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71c290-e4be-4674-a56a-ecf4c619f70c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:50.664438 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-03645078-8fdb-4d3f-b3e3-3c4a06a05317 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:50.756130 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03645078-8fdb-4d3f-b3e3-3c4a06a05317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:50.757831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03645078-8fdb-4d3f-b3e3-3c4a06a05317 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:50.767528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1132/2282] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:50.770881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1151/2283] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:50 2022] POST /identity/v3/users => generated 292 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:50.775152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03645078-8fdb-4d3f-b3e3-3c4a06a05317 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:50.790607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03645078-8fdb-4d3f-b3e3-3c4a06a05317 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:50.796702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74932ea7-ab4a-4b53-a8ba-5112e0cbaaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:50.797870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74932ea7-ab4a-4b53-a8ba-5112e0cbaaee admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:50.798266 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74932ea7-ab4a-4b53-a8ba-5112e0cbaaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:50.798912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74932ea7-ab4a-4b53-a8ba-5112e0cbaaee admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:50.800492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74932ea7-ab4a-4b53-a8ba-5112e0cbaaee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:50.802497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74932ea7-ab4a-4b53-a8ba-5112e0cbaaee admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:50.815888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1133/2284] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:50.820001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71c290-e4be-4674-a56a-ecf4c619f70c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:50.839830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71c290-e4be-4674-a56a-ecf4c619f70c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:50.845771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd71c1-ee7d-4afa-9fea-42c1bd71fa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:50.847120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd71c1-ee7d-4afa-9fea-42c1bd71fa13 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:50.847511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd71c1-ee7d-4afa-9fea-42c1bd71fa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:50.847896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd71c1-ee7d-4afa-9fea-42c1bd71fa13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:50.848775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd71c1-ee7d-4afa-9fea-42c1bd71fa13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:50.850765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd71c1-ee7d-4afa-9fea-42c1bd71fa13 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:50.863926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1152/2285] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:50.870451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74932ea7-ab4a-4b53-a8ba-5112e0cbaaee admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:50.884958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74932ea7-ab4a-4b53-a8ba-5112e0cbaaee admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:50.890692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:50.895275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ee335-f542-4542-a5a3-ff0e9f89d4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:50.895275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ee335-f542-4542-a5a3-ff0e9f89d4eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:50.895275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ee335-f542-4542-a5a3-ff0e9f89d4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:50.895275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ee335-f542-4542-a5a3-ff0e9f89d4eb admin admin] PATH_INFO: `/v3/projects/d022b16653f241ec9c4044d89b01edd0/users/239188fb82d84e04b3411f0f97fbf0c9/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:50.944368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ee335-f542-4542-a5a3-ff0e9f89d4eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d022b16653f241ec9c4044d89b01edd0, user_id=239188fb82d84e04b3411f0f97fbf0c9, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:50.948686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ee335-f542-4542-a5a3-ff0e9f89d4eb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:51.055395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1134/2286] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:50 2022] PUT /identity/v3/projects/d022b16653f241ec9c4044d89b01edd0/users/239188fb82d84e04b3411f0f97fbf0c9/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:51.055827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd71c1-ee7d-4afa-9fea-42c1bd71fa13 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:51.055827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a8fd71c1-ee7d-4afa-9fea-42c1bd71fa13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:51.059460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a183d-bfcb-4326-905c-8e3eda3635a3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:51.059756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a183d-bfcb-4326-905c-8e3eda3635a3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:51.060037 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a183d-bfcb-4326-905c-8e3eda3635a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:51.072983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd71c1-ee7d-4afa-9fea-42c1bd71fa13 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:51.113197 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b40a183d-bfcb-4326-905c-8e3eda3635a3 None None] MFA Rules not processed for user `239188fb82d84e04b3411f0f97fbf0c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:51.114474 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b40a183d-bfcb-4326-905c-8e3eda3635a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:51.153775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:51.155004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:51.155290 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:51.155918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] PATH_INFO: `/v3/users/baaf254c2b28444bab7f1c7cc3fbc42c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:51.156959 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baaf254c2b28444bab7f1c7cc3fbc42c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:51.159945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:51.221015 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1135/2287] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:51.234108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a183d-bfcb-4326-905c-8e3eda3635a3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:51.235181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b40a183d-bfcb-4326-905c-8e3eda3635a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:51.319041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baaf254c2b28444bab7f1c7cc3fbc42c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:51.322915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a183d-bfcb-4326-905c-8e3eda3635a3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:51.334085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baaf254c2b28444bab7f1c7cc3fbc42c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:51.336299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e346313-ef85-40dc-bc2d-bc4283d3c5f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:51.337073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e346313-ef85-40dc-bc2d-bc4283d3c5f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:51.337435 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e346313-ef85-40dc-bc2d-bc4283d3c5f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:51.337788 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e346313-ef85-40dc-bc2d-bc4283d3c5f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:51.342126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baaf254c2b28444bab7f1c7cc3fbc42c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:51.367741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e346313-ef85-40dc-bc2d-bc4283d3c5f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:51.369160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e346313-ef85-40dc-bc2d-bc4283d3c5f1 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:51.392597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1136/2288] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:51:51.410326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e346313-ef85-40dc-bc2d-bc4283d3c5f1 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:51.426348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e346313-ef85-40dc-bc2d-bc4283d3c5f1 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:51.431934 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baaf254c2b28444bab7f1c7cc3fbc42c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:51.431934 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baaf254c2b28444bab7f1c7cc3fbc42c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:51.444152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1153/2289] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:51 2022] DELETE /identity/v3/users/baaf254c2b28444bab7f1c7cc3fbc42c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:51.447416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d044f-287a-4cda-bc66-251571ca31e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:51.449826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d044f-287a-4cda-bc66-251571ca31e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:51.450279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d044f-287a-4cda-bc66-251571ca31e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:51.450723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d044f-287a-4cda-bc66-251571ca31e8 admin admin] PATH_INFO: `/v3/users/239188fb82d84e04b3411f0f97fbf0c9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:51.453157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d044f-287a-4cda-bc66-251571ca31e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=239188fb82d84e04b3411f0f97fbf0c9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:51.455751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d044f-287a-4cda-bc66-251571ca31e8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:51.461073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:51.738923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-907d044f-287a-4cda-bc66-251571ca31e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '239188fb82d84e04b3411f0f97fbf0c9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:51.763110 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:51.765343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:51.769786 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-907d044f-287a-4cda-bc66-251571ca31e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '239188fb82d84e04b3411f0f97fbf0c9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:51.793012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-907d044f-287a-4cda-bc66-251571ca31e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '239188fb82d84e04b3411f0f97fbf0c9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:51.861276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bd5d8-459f-497d-9968-63015e51cc26 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:51.866613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:51.870990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:51.870990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:51.870990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] PATH_INFO: `/v3/projects/b61afdae54fc4f76b7dd31798c4226f4` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:51.870990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b61afdae54fc4f76b7dd31798c4226f4)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:51.871361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:51.911507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-907d044f-287a-4cda-bc66-251571ca31e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '239188fb82d84e04b3411f0f97fbf0c9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:51.912534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-907d044f-287a-4cda-bc66-251571ca31e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '239188fb82d84e04b3411f0f97fbf0c9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:51.921406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1137/2290] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:51 2022] DELETE /identity/v3/users/239188fb82d84e04b3411f0f97fbf0c9 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:51.963731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f25277-f5fa-4cb0-93d6-64edb4b18200 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:51.965569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f25277-f5fa-4cb0-93d6-64edb4b18200 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:51.965569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f25277-f5fa-4cb0-93d6-64edb4b18200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:51.998961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a7f25277-f5fa-4cb0-93d6-64edb4b18200 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:52.000608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a7f25277-f5fa-4cb0-93d6-64edb4b18200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:52.006687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b61afdae54fc4f76b7dd31798c4226f4', 'request_id': 'req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:52.077420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b61afdae54fc4f76b7dd31798c4226f4', 'request_id': 'req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:52.084503 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b61afdae54fc4f76b7dd31798c4226f4', 'request_id': 'req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:52.097779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1138/2291] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:52.117793 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f25277-f5fa-4cb0-93d6-64edb4b18200 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:52.119444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a7f25277-f5fa-4cb0-93d6-64edb4b18200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:52.121069 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] The token cache is being invalidate because project b61afdae54fc4f76b7dd31798c4226f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:52.121412 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:52.122377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-cd727ed9-f2c2-47c4-b2dc-4918b6f0bf7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:52.126536 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1154/2292] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:51 2022] DELETE /identity/v3/projects/b61afdae54fc4f76b7dd31798c4226f4 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:52.133869 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691e993-249c-4dcd-a1fc-abe593213424 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:52.134163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691e993-249c-4dcd-a1fc-abe593213424 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:52.134654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691e993-249c-4dcd-a1fc-abe593213424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:52.182975 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a691e993-249c-4dcd-a1fc-abe593213424 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:52.186599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a691e993-249c-4dcd-a1fc-abe593213424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:52.232097 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1155/2293] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:52.243011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede020a2-55dc-4e4e-8b11-db2f2d650725 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:52.243519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede020a2-55dc-4e4e-8b11-db2f2d650725 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:52.244066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede020a2-55dc-4e4e-8b11-db2f2d650725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:52.292343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ede020a2-55dc-4e4e-8b11-db2f2d650725 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:52.293883 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ede020a2-55dc-4e4e-8b11-db2f2d650725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:52.311485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f25277-f5fa-4cb0-93d6-64edb4b18200 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:52.318075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1156/2294] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:52.322026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede020a2-55dc-4e4e-8b11-db2f2d650725 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:52.325650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d45cf6-a25a-4b25-a737-8ef0721494ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:52.329255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45cf6-a25a-4b25-a737-8ef0721494ba service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:52.336109 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45cf6-a25a-4b25-a737-8ef0721494ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:52.337175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45cf6-a25a-4b25-a737-8ef0721494ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:52.344521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-16d45cf6-a25a-4b25-a737-8ef0721494ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:52.355542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede020a2-55dc-4e4e-8b11-db2f2d650725 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:52.379859 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9226fe-0e9e-4de6-897b-31f3570f96cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:52.380632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9226fe-0e9e-4de6-897b-31f3570f96cb admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:52.380945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9226fe-0e9e-4de6-897b-31f3570f96cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:52.387983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9226fe-0e9e-4de6-897b-31f3570f96cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:52.387983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9226fe-0e9e-4de6-897b-31f3570f96cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:52.387983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9226fe-0e9e-4de6-897b-31f3570f96cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:52.387983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9226fe-0e9e-4de6-897b-31f3570f96cb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:52.389717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d45cf6-a25a-4b25-a737-8ef0721494ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:52.391219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d45cf6-a25a-4b25-a737-8ef0721494ba service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:52.415150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1139/2296] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:52.422563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1157/2296] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:51:52.422715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d45cf6-a25a-4b25-a737-8ef0721494ba service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:52.428486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d45cf6-a25a-4b25-a737-8ef0721494ba service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:52.439878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7aa9e-f60c-48e0-910c-ee6a5922fc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:52.439878 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7aa9e-f60c-48e0-910c-ee6a5922fc8f admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:52.440285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7aa9e-f60c-48e0-910c-ee6a5922fc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:52.440734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7aa9e-f60c-48e0-910c-ee6a5922fc8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:52.441584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7aa9e-f60c-48e0-910c-ee6a5922fc8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:52.443759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7aa9e-f60c-48e0-910c-ee6a5922fc8f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:52.615708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1140/2297] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:52 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:52.619235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9226fe-0e9e-4de6-897b-31f3570f96cb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:52.630840 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9226fe-0e9e-4de6-897b-31f3570f96cb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:52.741757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72ee4e-9f6b-4e3b-87cb-5e0aff303ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:52.743460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72ee4e-9f6b-4e3b-87cb-5e0aff303ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:52.743460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72ee4e-9f6b-4e3b-87cb-5e0aff303ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:52.743460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72ee4e-9f6b-4e3b-87cb-5e0aff303ebe admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:52.743883 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72ee4e-9f6b-4e3b-87cb-5e0aff303ebe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:52.745927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72ee4e-9f6b-4e3b-87cb-5e0aff303ebe admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:52.826179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1158/2298] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:52 2022] POST /identity/v3/users => generated 287 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:52.830940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7aa9e-f60c-48e0-910c-ee6a5922fc8f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:52.842481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7aa9e-f60c-48e0-910c-ee6a5922fc8f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:52.848447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e121e92-ed22-4ecf-bc3b-907daaa9277d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:52.849295 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e121e92-ed22-4ecf-bc3b-907daaa9277d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:52.849610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e121e92-ed22-4ecf-bc3b-907daaa9277d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:52.849941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e121e92-ed22-4ecf-bc3b-907daaa9277d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:52.850702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e121e92-ed22-4ecf-bc3b-907daaa9277d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:52.855448 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e121e92-ed22-4ecf-bc3b-907daaa9277d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:52.887234 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72ee4e-9f6b-4e3b-87cb-5e0aff303ebe admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:52.887354 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:52.887798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1141/2299] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:52.897060 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72ee4e-9f6b-4e3b-87cb-5e0aff303ebe admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:52.905670 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c20ee-6197-4dc4-bc0d-c3286accc871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:52.906764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c20ee-6197-4dc4-bc0d-c3286accc871 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:52.907257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c20ee-6197-4dc4-bc0d-c3286accc871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:52.913154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c20ee-6197-4dc4-bc0d-c3286accc871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:52.921851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c20ee-6197-4dc4-bc0d-c3286accc871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:52.921851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c20ee-6197-4dc4-bc0d-c3286accc871 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:52.950941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1159/2300] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:52.956437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e121e92-ed22-4ecf-bc3b-907daaa9277d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:52.969757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e121e92-ed22-4ecf-bc3b-907daaa9277d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:52.978612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508039f-a481-477f-a6e6-b905a50f05a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:52.978612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508039f-a481-477f-a6e6-b905a50f05a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:52.978612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508039f-a481-477f-a6e6-b905a50f05a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:52.978929 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508039f-a481-477f-a6e6-b905a50f05a7 admin admin] PATH_INFO: `/v3/projects/a9024588293448c3ac92cc12a5f24d76/users/c14b337eed0e4877bf6dcba04724f9f5/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:53.007090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508039f-a481-477f-a6e6-b905a50f05a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9024588293448c3ac92cc12a5f24d76, user_id=c14b337eed0e4877bf6dcba04724f9f5, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:53.014581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508039f-a481-477f-a6e6-b905a50f05a7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:53.086436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1142/2301] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:52 2022] PUT /identity/v3/projects/a9024588293448c3ac92cc12a5f24d76/users/c14b337eed0e4877bf6dcba04724f9f5/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:53.096796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923c728-7d9b-49fb-a51c-c488decdd969 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:53.097376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923c728-7d9b-49fb-a51c-c488decdd969 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:53.097455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923c728-7d9b-49fb-a51c-c488decdd969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:53.126899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d923c728-7d9b-49fb-a51c-c488decdd969 None None] MFA Rules not processed for user `c14b337eed0e4877bf6dcba04724f9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:53.128790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d923c728-7d9b-49fb-a51c-c488decdd969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:53.169772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508039f-a481-477f-a6e6-b905a50f05a7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:53.172393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d508039f-a481-477f-a6e6-b905a50f05a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:53.233937 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1160/2302] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:53.240041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923c728-7d9b-49fb-a51c-c488decdd969 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:53.359724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:53.382639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508039f-a481-477f-a6e6-b905a50f05a7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:53.393409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:53.393409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:53.393409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:53.393409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] PATH_INFO: `/v3/projects/d022b16653f241ec9c4044d89b01edd0` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:53.393831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d022b16653f241ec9c4044d89b01edd0)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:53.395307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:53.420279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923c728-7d9b-49fb-a51c-c488decdd969 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:53.433794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c045d94-f69f-4395-84fa-c04e3bc87927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:53.434775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c045d94-f69f-4395-84fa-c04e3bc87927 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:53.435261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c045d94-f69f-4395-84fa-c04e3bc87927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:53.435763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c045d94-f69f-4395-84fa-c04e3bc87927 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:53.439308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c045d94-f69f-4395-84fa-c04e3bc87927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:53.444189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c045d94-f69f-4395-84fa-c04e3bc87927 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:53.593959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd022b16653f241ec9c4044d89b01edd0', 'request_id': 'req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:53.605532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd022b16653f241ec9c4044d89b01edd0', 'request_id': 'req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:53.625099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd022b16653f241ec9c4044d89b01edd0', 'request_id': 'req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:53.631425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1161/2303] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:53 2022] POST /identity/v3/projects => generated 371 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:53.635598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c045d94-f69f-4395-84fa-c04e3bc87927 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:53.702341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] The token cache is being invalidate because project d022b16653f241ec9c4044d89b01edd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:51:53.707906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:53.709261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-87e71bce-0fa4-48a7-8f09-3e245e43cc3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:53.720858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1143/2304] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:51:53 2022] DELETE /identity/v3/projects/d022b16653f241ec9c4044d89b01edd0 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:53.720858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee76af1-5983-4d1e-98c5-ccde56e293db None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:53.720858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee76af1-5983-4d1e-98c5-ccde56e293db None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:53.721703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee76af1-5983-4d1e-98c5-ccde56e293db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:53.805167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-cee76af1-5983-4d1e-98c5-ccde56e293db None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:53.807279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-cee76af1-5983-4d1e-98c5-ccde56e293db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:53.842773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c045d94-f69f-4395-84fa-c04e3bc87927 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:53.851278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf759597-53e8-4361-b552-f0925b01be39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:53.851278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf759597-53e8-4361-b552-f0925b01be39 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:53.851278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf759597-53e8-4361-b552-f0925b01be39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:53.851278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf759597-53e8-4361-b552-f0925b01be39 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:53.852296 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf759597-53e8-4361-b552-f0925b01be39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:53.854556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf759597-53e8-4361-b552-f0925b01be39 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:53.895083 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1144/2305] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:53.896646 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272dd0b-bf93-46d4-b567-a79bf6185394 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:53.896966 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272dd0b-bf93-46d4-b567-a79bf6185394 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:53.897252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272dd0b-bf93-46d4-b567-a79bf6185394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:53.920402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8272dd0b-bf93-46d4-b567-a79bf6185394 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:53.921934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8272dd0b-bf93-46d4-b567-a79bf6185394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:53.931166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1145/2306] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:53.935841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272dd0b-bf93-46d4-b567-a79bf6185394 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:53.946745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272dd0b-bf93-46d4-b567-a79bf6185394 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:53.962213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1162/2307] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:53 2022] POST /identity/v3/users => generated 292 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:53.968090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153662fe-d1d4-4918-9542-5505d2090f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:53.969086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf759597-53e8-4361-b552-f0925b01be39 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:53.971045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cf759597-53e8-4361-b552-f0925b01be39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:53.973261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153662fe-d1d4-4918-9542-5505d2090f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:53.973737 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153662fe-d1d4-4918-9542-5505d2090f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:53.973879 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153662fe-d1d4-4918-9542-5505d2090f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:53.974749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153662fe-d1d4-4918-9542-5505d2090f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:51:53.975063 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153662fe-d1d4-4918-9542-5505d2090f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:53.978240 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153662fe-d1d4-4918-9542-5505d2090f1d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:53.986049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf759597-53e8-4361-b552-f0925b01be39 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:53.992914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243c954-8608-4dde-b2d4-27e6f1d0b10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:53.993683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243c954-8608-4dde-b2d4-27e6f1d0b10d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:53.993967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243c954-8608-4dde-b2d4-27e6f1d0b10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:53.994303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243c954-8608-4dde-b2d4-27e6f1d0b10d admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:53.995085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243c954-8608-4dde-b2d4-27e6f1d0b10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:53.996798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243c954-8608-4dde-b2d4-27e6f1d0b10d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:54.009115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1146/2308] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:51:54.009115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153662fe-d1d4-4918-9542-5505d2090f1d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:54.012915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1163/2309] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:54.015910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243c954-8608-4dde-b2d4-27e6f1d0b10d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:54.029702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243c954-8608-4dde-b2d4-27e6f1d0b10d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:54.037938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10ca66-2c70-4ae9-8a84-6e585b19f815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:54.039225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ca66-2c70-4ae9-8a84-6e585b19f815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.039654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ca66-2c70-4ae9-8a84-6e585b19f815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.040222 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10ca66-2c70-4ae9-8a84-6e585b19f815 admin admin] PATH_INFO: `/v3/projects/003dde174e54459bbd881966bf2577fa/users/a60b2e1c43c04b698a8fd1e1712f52ef/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.042905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:54.048155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153662fe-d1d4-4918-9542-5505d2090f1d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:54.052633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31947721-914c-443e-aa99-5d596595ea20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:54.053413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31947721-914c-443e-aa99-5d596595ea20 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.053702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31947721-914c-443e-aa99-5d596595ea20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.053982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31947721-914c-443e-aa99-5d596595ea20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.054681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31947721-914c-443e-aa99-5d596595ea20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:54.056938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31947721-914c-443e-aa99-5d596595ea20 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:54.061773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10ca66-2c70-4ae9-8a84-6e585b19f815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=003dde174e54459bbd881966bf2577fa, user_id=a60b2e1c43c04b698a8fd1e1712f52ef, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:54.067046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10ca66-2c70-4ae9-8a84-6e585b19f815 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:54.144317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1164/2310] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:54 2022] PUT /identity/v3/projects/003dde174e54459bbd881966bf2577fa/users/a60b2e1c43c04b698a8fd1e1712f52ef/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:51:54.148845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10ca66-2c70-4ae9-8a84-6e585b19f815 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:54.151155 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1147/2311] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:51:54 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:54.155595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31947721-914c-443e-aa99-5d596595ea20 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:54.248948 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10ca66-2c70-4ae9-8a84-6e585b19f815 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:54.270015 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31947721-914c-443e-aa99-5d596595ea20 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:54.279373 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e8db9-985d-4d1d-8f97-7d36c150ec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:54.280259 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e8db9-985d-4d1d-8f97-7d36c150ec47 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.282503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e8db9-985d-4d1d-8f97-7d36c150ec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.282503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e8db9-985d-4d1d-8f97-7d36c150ec47 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.282503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e8db9-985d-4d1d-8f97-7d36c150ec47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:54.284850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e8db9-985d-4d1d-8f97-7d36c150ec47 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:54.329006 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21175ab1-935e-4905-b1ab-a611817e6148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:54.330051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21175ab1-935e-4905-b1ab-a611817e6148 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.331161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21175ab1-935e-4905-b1ab-a611817e6148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.331546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21175ab1-935e-4905-b1ab-a611817e6148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.332776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21175ab1-935e-4905-b1ab-a611817e6148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:54.335455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21175ab1-935e-4905-b1ab-a611817e6148 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:54.349755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1165/2312] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:54.354196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21175ab1-935e-4905-b1ab-a611817e6148 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:54.368293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21175ab1-935e-4905-b1ab-a611817e6148 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:54.374746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642755d2-bdf7-4952-a8a7-514917688bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:54.375771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642755d2-bdf7-4952-a8a7-514917688bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.379635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642755d2-bdf7-4952-a8a7-514917688bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.379635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642755d2-bdf7-4952-a8a7-514917688bb9 admin admin] PATH_INFO: `/v3/projects/003dde174e54459bbd881966bf2577fa/users/a60b2e1c43c04b698a8fd1e1712f52ef/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.379635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642755d2-bdf7-4952-a8a7-514917688bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=003dde174e54459bbd881966bf2577fa, user_id=a60b2e1c43c04b698a8fd1e1712f52ef, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:54.383518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642755d2-bdf7-4952-a8a7-514917688bb9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:54.402742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1166/2313] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:54 2022] PUT /identity/v3/projects/003dde174e54459bbd881966bf2577fa/users/a60b2e1c43c04b698a8fd1e1712f52ef/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:54.409097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1148/2314] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:51:54 2022] POST /identity/v3/users => generated 288 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:51:54.420496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642755d2-bdf7-4952-a8a7-514917688bb9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:54.427902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf098f-051d-414e-9925-5fb32b9d2811 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.431473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf098f-051d-414e-9925-5fb32b9d2811 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.431473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf098f-051d-414e-9925-5fb32b9d2811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.472820 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-33cf098f-051d-414e-9925-5fb32b9d2811 None None] MFA Rules not processed for user `a60b2e1c43c04b698a8fd1e1712f52ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:54.483759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-33cf098f-051d-414e-9925-5fb32b9d2811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:54.520923 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:54.529634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642755d2-bdf7-4952-a8a7-514917688bb9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:54.534323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475882b-9844-453a-9912-b0265c2ceb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:54.535021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475882b-9844-453a-9912-b0265c2ceb70 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.535310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475882b-9844-453a-9912-b0265c2ceb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.535582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475882b-9844-453a-9912-b0265c2ceb70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.536331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475882b-9844-453a-9912-b0265c2ceb70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:54.538351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475882b-9844-453a-9912-b0265c2ceb70 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:54.554708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1167/2315] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:54.556138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a540a19-7bc3-4ef3-94f5-6e4748c6eec7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.556441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a540a19-7bc3-4ef3-94f5-6e4748c6eec7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.556949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a540a19-7bc3-4ef3-94f5-6e4748c6eec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.577486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0a540a19-7bc3-4ef3-94f5-6e4748c6eec7 None None] MFA Rules not processed for user `8016e561397f4b4faca9e03343d51355`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:54.579460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0a540a19-7bc3-4ef3-94f5-6e4748c6eec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:54.582628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:54.619482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1149/2316] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:54 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:54.620308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf098f-051d-414e-9925-5fb32b9d2811 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:54.632501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf098f-051d-414e-9925-5fb32b9d2811 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:54.637349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d2836-2cbf-4d72-89a4-7fc7c0c6c757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:54.638021 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d2836-2cbf-4d72-89a4-7fc7c0c6c757 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.638355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d2836-2cbf-4d72-89a4-7fc7c0c6c757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.638669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d2836-2cbf-4d72-89a4-7fc7c0c6c757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.639467 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d2836-2cbf-4d72-89a4-7fc7c0c6c757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:54.641189 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d2836-2cbf-4d72-89a4-7fc7c0c6c757 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:54.656010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1150/2317] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:51:54.662518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1168/2318] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:54.662690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d2836-2cbf-4d72-89a4-7fc7c0c6c757 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:54.662690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-991d2836-2cbf-4d72-89a4-7fc7c0c6c757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:54.662886 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a540a19-7bc3-4ef3-94f5-6e4748c6eec7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:54.675037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a540a19-7bc3-4ef3-94f5-6e4748c6eec7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:54.679909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f0197-ba45-4d1c-903c-e3c97d1c0c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:54.682331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f0197-ba45-4d1c-903c-e3c97d1c0c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.682331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f0197-ba45-4d1c-903c-e3c97d1c0c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.682331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f0197-ba45-4d1c-903c-e3c97d1c0c2c admin admin] PATH_INFO: `/v3/projects/00a2700946794bd7bad3d61b1dce0a5f/users/7948dfb2bba7418d8df2cd7f03ee6373/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.703682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:54.707735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f0197-ba45-4d1c-903c-e3c97d1c0c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a2700946794bd7bad3d61b1dce0a5f, user_id=7948dfb2bba7418d8df2cd7f03ee6373, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:54.710874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f0197-ba45-4d1c-903c-e3c97d1c0c2c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:54.736962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d2836-2cbf-4d72-89a4-7fc7c0c6c757 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:54.741241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad61d7-4630-41da-a1f7-ef7ebecf9961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:54.741923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad61d7-4630-41da-a1f7-ef7ebecf9961 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.742203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad61d7-4630-41da-a1f7-ef7ebecf9961 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.743230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad61d7-4630-41da-a1f7-ef7ebecf9961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.757759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad61d7-4630-41da-a1f7-ef7ebecf9961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:54.757759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad61d7-4630-41da-a1f7-ef7ebecf9961 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:54.768540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1151/2319] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:54.799700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1169/2320] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:51:54 2022] PUT /identity/v3/projects/00a2700946794bd7bad3d61b1dce0a5f/users/7948dfb2bba7418d8df2cd7f03ee6373/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:51:54.810162 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e7f94-c598-4fe5-9ccc-2919d76d34a2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:54.817876 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e7f94-c598-4fe5-9ccc-2919d76d34a2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:54.817876 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e7f94-c598-4fe5-9ccc-2919d76d34a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:54.849741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-301e7f94-c598-4fe5-9ccc-2919d76d34a2 None None] MFA Rules not processed for user `7948dfb2bba7418d8df2cd7f03ee6373`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:51:54.850964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-301e7f94-c598-4fe5-9ccc-2919d76d34a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:54.956131 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1152/2321] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:51:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:51:55.108324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f0197-ba45-4d1c-903c-e3c97d1c0c2c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:55.171217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f0197-ba45-4d1c-903c-e3c97d1c0c2c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:55.176247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775db0a0-5103-41ac-836f-f80d4f419ef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:55.177176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775db0a0-5103-41ac-836f-f80d4f419ef4 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:55.177425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775db0a0-5103-41ac-836f-f80d4f419ef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:55.177705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775db0a0-5103-41ac-836f-f80d4f419ef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:55.195220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775db0a0-5103-41ac-836f-f80d4f419ef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:55.196708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775db0a0-5103-41ac-836f-f80d4f419ef4 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:55.208337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1170/2322] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:55.518085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:55.706542 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e7f94-c598-4fe5-9ccc-2919d76d34a2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:55.708470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-301e7f94-c598-4fe5-9ccc-2919d76d34a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:55.723509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e7f94-c598-4fe5-9ccc-2919d76d34a2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:55.727224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1003a-fdeb-4bbc-a6c4-bdb8a228c2c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:55.727988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1003a-fdeb-4bbc-a6c4-bdb8a228c2c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:55.728407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1003a-fdeb-4bbc-a6c4-bdb8a228c2c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:55.729203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1003a-fdeb-4bbc-a6c4-bdb8a228c2c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:55.730989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a2f1003a-fdeb-4bbc-a6c4-bdb8a228c2c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:55.827704 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1003a-fdeb-4bbc-a6c4-bdb8a228c2c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:55.829869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1003a-fdeb-4bbc-a6c4-bdb8a228c2c9 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:55.840824 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1153/2323] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:51:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:51:55.895292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:56.707294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:57.123096 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:57.419617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:58.290220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:51:59.622205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775db0a0-5103-41ac-836f-f80d4f419ef4 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:51:59.623671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-775db0a0-5103-41ac-836f-f80d4f419ef4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:51:59.700895 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775db0a0-5103-41ac-836f-f80d4f419ef4 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:51:59.706208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:51:59.707145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:51:59.707446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:51:59.707705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] PATH_INFO: `/v3/users/1625659dc8294a7583fc566d8397bbed` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:51:59.709091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1625659dc8294a7583fc566d8397bbed)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:51:59.711298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:51:59.975175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1625659dc8294a7583fc566d8397bbed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:59.990726 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1625659dc8294a7583fc566d8397bbed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:51:59.998106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1625659dc8294a7583fc566d8397bbed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:00.076147 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1625659dc8294a7583fc566d8397bbed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:00.076806 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1625659dc8294a7583fc566d8397bbed'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:00.085585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1171/2324] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:51:59 2022] DELETE /identity/v3/users/1625659dc8294a7583fc566d8397bbed => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:00.090654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d5914-7fd3-44ba-9bc4-e13ff8af09c4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:00.097629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d5914-7fd3-44ba-9bc4-e13ff8af09c4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:00.097629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d5914-7fd3-44ba-9bc4-e13ff8af09c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:00.118982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-699d5914-7fd3-44ba-9bc4-e13ff8af09c4 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:00.120309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-699d5914-7fd3-44ba-9bc4-e13ff8af09c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:00.158063 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:00.189805 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1154/2325] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:00.199760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:00.201352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:00.302819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a36ec-5ac6-4d56-aa30-7b38c8885b1f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:00.310531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145203e-c970-47cb-b5c2-c1567c36e056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:00.311204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145203e-c970-47cb-b5c2-c1567c36e056 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:00.311491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145203e-c970-47cb-b5c2-c1567c36e056 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:00.311769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145203e-c970-47cb-b5c2-c1567c36e056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:00.323867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145203e-c970-47cb-b5c2-c1567c36e056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:00.327979 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145203e-c970-47cb-b5c2-c1567c36e056 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:00.343646 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1172/2326] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:52:00.814993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d5914-7fd3-44ba-9bc4-e13ff8af09c4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:00.816481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-699d5914-7fd3-44ba-9bc4-e13ff8af09c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:00.834122 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d5914-7fd3-44ba-9bc4-e13ff8af09c4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:00.834122 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f716cf-46e2-4e14-a6b8-b3fd111fe76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:00.834900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f716cf-46e2-4e14-a6b8-b3fd111fe76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:00.835181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f716cf-46e2-4e14-a6b8-b3fd111fe76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:00.835713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f716cf-46e2-4e14-a6b8-b3fd111fe76b admin admin] PATH_INFO: `/v3/users/7948dfb2bba7418d8df2cd7f03ee6373` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:00.836990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f716cf-46e2-4e14-a6b8-b3fd111fe76b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7948dfb2bba7418d8df2cd7f03ee6373)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:00.838986 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f716cf-46e2-4e14-a6b8-b3fd111fe76b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:00.861716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145203e-c970-47cb-b5c2-c1567c36e056 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:00.863440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e145203e-c970-47cb-b5c2-c1567c36e056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:00.888705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145203e-c970-47cb-b5c2-c1567c36e056 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:00.900185 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:00.901278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:00.901624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:00.909288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] PATH_INFO: `/v3/projects/a2547613e85441299690cab07f6b23f2` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:00.909288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2547613e85441299690cab07f6b23f2)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:00.909288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:00.909288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:01.082328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2547613e85441299690cab07f6b23f2', 'request_id': 'req-b2d0fc42-6720-4519-a658-5672a52ed1f6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:01.088426 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-24f716cf-46e2-4e14-a6b8-b3fd111fe76b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7948dfb2bba7418d8df2cd7f03ee6373'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:01.098804 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-24f716cf-46e2-4e14-a6b8-b3fd111fe76b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7948dfb2bba7418d8df2cd7f03ee6373'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:01.112119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-24f716cf-46e2-4e14-a6b8-b3fd111fe76b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7948dfb2bba7418d8df2cd7f03ee6373'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:01.179321 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2547613e85441299690cab07f6b23f2', 'request_id': 'req-b2d0fc42-6720-4519-a658-5672a52ed1f6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:01.185697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2547613e85441299690cab07f6b23f2', 'request_id': 'req-b2d0fc42-6720-4519-a658-5672a52ed1f6'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:01.199984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] The token cache is being invalidate because project a2547613e85441299690cab07f6b23f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:52:01.200914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:01.201797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b2d0fc42-6720-4519-a658-5672a52ed1f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:01.203612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1173/2327] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:52:00 2022] DELETE /identity/v3/projects/a2547613e85441299690cab07f6b23f2 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:01.208929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a80165-65e5-46dc-9702-3e9d5f5cb376 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:01.209494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a80165-65e5-46dc-9702-3e9d5f5cb376 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:01.210070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a80165-65e5-46dc-9702-3e9d5f5cb376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:01.229278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-12a80165-65e5-46dc-9702-3e9d5f5cb376 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:01.230504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-12a80165-65e5-46dc-9702-3e9d5f5cb376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:01.231019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-24f716cf-46e2-4e14-a6b8-b3fd111fe76b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7948dfb2bba7418d8df2cd7f03ee6373'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:01.231822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-24f716cf-46e2-4e14-a6b8-b3fd111fe76b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7948dfb2bba7418d8df2cd7f03ee6373'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:01.244355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1155/2328] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:52:00 2022] DELETE /identity/v3/users/7948dfb2bba7418d8df2cd7f03ee6373 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:01.247448 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad390f-4612-4ae7-8ccf-f44d6f7c60f3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:01.247965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad390f-4612-4ae7-8ccf-f44d6f7c60f3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:01.254211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad390f-4612-4ae7-8ccf-f44d6f7c60f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:01.273533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b1ad390f-4612-4ae7-8ccf-f44d6f7c60f3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:01.274779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b1ad390f-4612-4ae7-8ccf-f44d6f7c60f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:01.309903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1174/2329] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:01.311634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bee45e-5324-4679-80b7-7a79433d3ceb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:01.312316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bee45e-5324-4679-80b7-7a79433d3ceb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:01.312607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bee45e-5324-4679-80b7-7a79433d3ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:01.342928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c7bee45e-5324-4679-80b7-7a79433d3ceb None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:01.344064 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c7bee45e-5324-4679-80b7-7a79433d3ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:01.348500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1156/2330] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:01.356574 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1175/2331] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:52:01.358896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad390f-4612-4ae7-8ccf-f44d6f7c60f3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:01.360487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b1ad390f-4612-4ae7-8ccf-f44d6f7c60f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:01.360632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bee45e-5324-4679-80b7-7a79433d3ceb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:01.376460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bee45e-5324-4679-80b7-7a79433d3ceb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:01.388428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4da95d-ec04-4243-a7f2-901079bd6d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:01.389252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4da95d-ec04-4243-a7f2-901079bd6d46 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:01.389701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4da95d-ec04-4243-a7f2-901079bd6d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:01.389986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4da95d-ec04-4243-a7f2-901079bd6d46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:01.390757 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4da95d-ec04-4243-a7f2-901079bd6d46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:52:01.391066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4da95d-ec04-4243-a7f2-901079bd6d46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:01.392804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4da95d-ec04-4243-a7f2-901079bd6d46 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:01.413953 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1176/2332] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:52:01.419108 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4da95d-ec04-4243-a7f2-901079bd6d46 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:01.442001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:01.442856 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4da95d-ec04-4243-a7f2-901079bd6d46 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:01.448033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ad390f-4612-4ae7-8ccf-f44d6f7c60f3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:01.449199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0465e16-4b82-42aa-a051-58439812e983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:01.450289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0465e16-4b82-42aa-a051-58439812e983 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:01.450790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0465e16-4b82-42aa-a051-58439812e983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:01.451230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0465e16-4b82-42aa-a051-58439812e983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:01.452033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afced819-9d9f-4aed-9536-8dc554160747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:01.452326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0465e16-4b82-42aa-a051-58439812e983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:01.452690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afced819-9d9f-4aed-9536-8dc554160747 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:01.453017 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afced819-9d9f-4aed-9536-8dc554160747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:01.453342 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afced819-9d9f-4aed-9536-8dc554160747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:01.455688 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0465e16-4b82-42aa-a051-58439812e983 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:01.465709 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afced819-9d9f-4aed-9536-8dc554160747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:01.468101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afced819-9d9f-4aed-9536-8dc554160747 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:01.480471 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1157/2333] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:52:01.556489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1177/2334] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:52:01 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:52:01.560972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afced819-9d9f-4aed-9536-8dc554160747 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:01.573029 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afced819-9d9f-4aed-9536-8dc554160747 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:01.632792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a2e37-0f6e-4623-8d34-abded7fb700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:01.633403 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a2e37-0f6e-4623-8d34-abded7fb700d admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:01.633687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a2e37-0f6e-4623-8d34-abded7fb700d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:01.633963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a2e37-0f6e-4623-8d34-abded7fb700d admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:01.634659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a2e37-0f6e-4623-8d34-abded7fb700d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:01.636585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a2e37-0f6e-4623-8d34-abded7fb700d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:01.757457 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0465e16-4b82-42aa-a051-58439812e983 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:01.758682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1158/2335] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:52:01 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:52:01.771976 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0465e16-4b82-42aa-a051-58439812e983 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:01.786477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9a930-6a7d-41f6-8487-9076c4d12aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:01.787178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9a930-6a7d-41f6-8487-9076c4d12aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:01.787460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9a930-6a7d-41f6-8487-9076c4d12aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:01.787731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9a930-6a7d-41f6-8487-9076c4d12aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:01.788469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9a930-6a7d-41f6-8487-9076c4d12aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:01.790347 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9a930-6a7d-41f6-8487-9076c4d12aa6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:01.801634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1178/2336] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:52:01.805533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a2e37-0f6e-4623-8d34-abded7fb700d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:01.819751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a2e37-0f6e-4623-8d34-abded7fb700d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:01.825719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d51803-9916-4869-98e5-60b6f8048507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:01.826834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d51803-9916-4869-98e5-60b6f8048507 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:01.827231 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d51803-9916-4869-98e5-60b6f8048507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:01.827710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d51803-9916-4869-98e5-60b6f8048507 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:01.828603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d51803-9916-4869-98e5-60b6f8048507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:01.830723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d51803-9916-4869-98e5-60b6f8048507 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:01.847918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1159/2337] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:52:01.852284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9a930-6a7d-41f6-8487-9076c4d12aa6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:01.864023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9a930-6a7d-41f6-8487-9076c4d12aa6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:01.873126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e664050-d6ec-4413-99b0-2ab055e7a153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:01.874049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e664050-d6ec-4413-99b0-2ab055e7a153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:01.874339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e664050-d6ec-4413-99b0-2ab055e7a153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:01.874620 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e664050-d6ec-4413-99b0-2ab055e7a153 admin admin] PATH_INFO: `/v3/projects/fce867e89c2e456996efceb5897f397f/users/4ab04c8cb3f14a359ffebe8ade37dd74/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:01.891890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e664050-d6ec-4413-99b0-2ab055e7a153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fce867e89c2e456996efceb5897f397f, user_id=4ab04c8cb3f14a359ffebe8ade37dd74, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:01.893093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e664050-d6ec-4413-99b0-2ab055e7a153 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:01.966999 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1179/2338] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:52:01 2022] PUT /identity/v3/projects/fce867e89c2e456996efceb5897f397f/users/4ab04c8cb3f14a359ffebe8ade37dd74/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:01.970299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d51803-9916-4869-98e5-60b6f8048507 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:01.973850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a4d51803-9916-4869-98e5-60b6f8048507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:01.977557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33477551-ae80-4d46-9b04-4831764b6033 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:01.977838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33477551-ae80-4d46-9b04-4831764b6033 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:01.978121 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33477551-ae80-4d46-9b04-4831764b6033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:01.996947 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d51803-9916-4869-98e5-60b6f8048507 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:02.007025 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:02.015378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-33477551-ae80-4d46-9b04-4831764b6033 None None] MFA Rules not processed for user `4ab04c8cb3f14a359ffebe8ade37dd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:02.017857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-33477551-ae80-4d46-9b04-4831764b6033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:02.072866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:02.074069 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:02.074360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:02.074639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] PATH_INFO: `/v3/projects/00a2700946794bd7bad3d61b1dce0a5f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:02.076020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a2700946794bd7bad3d61b1dce0a5f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:02.078437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:02.093394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1180/2339] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:02.102786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33477551-ae80-4d46-9b04-4831764b6033 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:02.104641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-33477551-ae80-4d46-9b04-4831764b6033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:02.172458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33477551-ae80-4d46-9b04-4831764b6033 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:02.198138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a2700946794bd7bad3d61b1dce0a5f', 'request_id': 'req-90835c47-6c43-499a-8d5c-c4b12aa78d8a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:02.204076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a2700946794bd7bad3d61b1dce0a5f', 'request_id': 'req-90835c47-6c43-499a-8d5c-c4b12aa78d8a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:02.215667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a2700946794bd7bad3d61b1dce0a5f', 'request_id': 'req-90835c47-6c43-499a-8d5c-c4b12aa78d8a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:02.242992 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305ba4e-6e76-4eca-ab01-a8419296c5f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:02.243805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305ba4e-6e76-4eca-ab01-a8419296c5f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:02.244122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305ba4e-6e76-4eca-ab01-a8419296c5f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:02.244396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305ba4e-6e76-4eca-ab01-a8419296c5f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:02.277701 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:02.282209 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] The token cache is being invalidate because project 00a2700946794bd7bad3d61b1dce0a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:52:02.282499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:02.283792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90835c47-6c43-499a-8d5c-c4b12aa78d8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:02.285430 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1160/2340] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:52:01 2022] DELETE /identity/v3/projects/00a2700946794bd7bad3d61b1dce0a5f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:02.290793 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020303b-65ef-419a-9d01-9ebfbf16afac None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:02.291167 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020303b-65ef-419a-9d01-9ebfbf16afac None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:02.291523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020303b-65ef-419a-9d01-9ebfbf16afac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:02.308558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305ba4e-6e76-4eca-ab01-a8419296c5f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:02.310231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305ba4e-6e76-4eca-ab01-a8419296c5f1 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:02.314139 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9020303b-65ef-419a-9d01-9ebfbf16afac None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:02.316107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9020303b-65ef-419a-9d01-9ebfbf16afac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:02.320787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1181/2341] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:52:02.395081 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1161/2342] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:52:02.469122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dc8a9-776e-484d-9670-dd7d7767389a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:02.469541 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dc8a9-776e-484d-9670-dd7d7767389a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:02.469838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dc8a9-776e-484d-9670-dd7d7767389a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:02.495790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e85dc8a9-776e-484d-9670-dd7d7767389a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:02.497828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e85dc8a9-776e-484d-9670-dd7d7767389a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:02.507831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1182/2343] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:02.512844 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020303b-65ef-419a-9d01-9ebfbf16afac None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:02.525333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020303b-65ef-419a-9d01-9ebfbf16afac None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:02.530373 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34ce07-80a1-4c13-b516-de814883394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:02.531046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34ce07-80a1-4c13-b516-de814883394f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:02.531330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34ce07-80a1-4c13-b516-de814883394f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:02.531608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34ce07-80a1-4c13-b516-de814883394f admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:02.532339 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34ce07-80a1-4c13-b516-de814883394f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:52:02.532760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34ce07-80a1-4c13-b516-de814883394f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:02.535233 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34ce07-80a1-4c13-b516-de814883394f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:02.550507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1162/2344] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:52:02.557455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dc8a9-776e-484d-9670-dd7d7767389a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:02.565848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dc8a9-776e-484d-9670-dd7d7767389a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:02.570925 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d522a-d0ff-4678-b003-17cd90f51dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:02.571658 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d522a-d0ff-4678-b003-17cd90f51dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:02.571946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d522a-d0ff-4678-b003-17cd90f51dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:02.572220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d522a-d0ff-4678-b003-17cd90f51dd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:02.572994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d522a-d0ff-4678-b003-17cd90f51dd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:02.574914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d522a-d0ff-4678-b003-17cd90f51dd1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:02.578487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:02.684111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1183/2345] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:52:02 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:52:02.688550 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34ce07-80a1-4c13-b516-de814883394f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:02.699910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34ce07-80a1-4c13-b516-de814883394f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:02.791973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6e845-34fe-4ba9-bfcc-10b3424fd8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:02.792668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e845-34fe-4ba9-bfcc-10b3424fd8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:02.792990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e845-34fe-4ba9-bfcc-10b3424fd8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:02.793270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6e845-34fe-4ba9-bfcc-10b3424fd8f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:02.793981 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6e845-34fe-4ba9-bfcc-10b3424fd8f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:02.796079 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6e845-34fe-4ba9-bfcc-10b3424fd8f9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:02.879124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1163/2346] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:52:02 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:52:02.880866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d522a-d0ff-4678-b003-17cd90f51dd1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:02.895082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d522a-d0ff-4678-b003-17cd90f51dd1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:02.902399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33e937-4a2b-4ced-927b-28196b784a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:02.902399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33e937-4a2b-4ced-927b-28196b784a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:02.902399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33e937-4a2b-4ced-927b-28196b784a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:02.902795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33e937-4a2b-4ced-927b-28196b784a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:02.903731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33e937-4a2b-4ced-927b-28196b784a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:02.905439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33e937-4a2b-4ced-927b-28196b784a87 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:02.917129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1184/2347] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:52:02.921409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6e845-34fe-4ba9-bfcc-10b3424fd8f9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:02.941017 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6e845-34fe-4ba9-bfcc-10b3424fd8f9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:02.946072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c6ac2-a6be-4252-a30e-05405d91dee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:02.946759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c6ac2-a6be-4252-a30e-05405d91dee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:02.947065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c6ac2-a6be-4252-a30e-05405d91dee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:02.947333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c6ac2-a6be-4252-a30e-05405d91dee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:02.948081 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c6ac2-a6be-4252-a30e-05405d91dee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:02.950263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c6ac2-a6be-4252-a30e-05405d91dee4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:02.963019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1164/2348] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:52:02.969145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33e937-4a2b-4ced-927b-28196b784a87 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:02.987316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33e937-4a2b-4ced-927b-28196b784a87 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:02.992969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe2c25-ea67-4eaa-b985-52767b331d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:02.993931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe2c25-ea67-4eaa-b985-52767b331d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:02.994245 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe2c25-ea67-4eaa-b985-52767b331d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:02.994916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe2c25-ea67-4eaa-b985-52767b331d82 admin admin] PATH_INFO: `/v3/projects/36517f59aa7e426e90ef1ecf7cf4a811/users/6472cd2e037d482db8c8047936132b73/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:03.014766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe2c25-ea67-4eaa-b985-52767b331d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36517f59aa7e426e90ef1ecf7cf4a811, user_id=6472cd2e037d482db8c8047936132b73, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:03.022680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe2c25-ea67-4eaa-b985-52767b331d82 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:03.107509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1185/2349] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:52:02 2022] PUT /identity/v3/projects/36517f59aa7e426e90ef1ecf7cf4a811/users/6472cd2e037d482db8c8047936132b73/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:52:03.125596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ff62-d963-493b-83b3-28ba1f2a5ab2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:03.125915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ff62-d963-493b-83b3-28ba1f2a5ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:03.126197 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036ff62-d963-493b-83b3-28ba1f2a5ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:03.155746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d036ff62-d963-493b-83b3-28ba1f2a5ab2 None None] MFA Rules not processed for user `6472cd2e037d482db8c8047936132b73`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:03.157059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d036ff62-d963-493b-83b3-28ba1f2a5ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:03.237840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:03.249064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1165/2350] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:03.254282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe2c25-ea67-4eaa-b985-52767b331d82 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:03.342650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe2c25-ea67-4eaa-b985-52767b331d82 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:03.348289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a40f11-85e8-437c-a8fb-707d1c7d4dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:03.348989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a40f11-85e8-437c-a8fb-707d1c7d4dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:03.349315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a40f11-85e8-437c-a8fb-707d1c7d4dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:03.349563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a40f11-85e8-437c-a8fb-707d1c7d4dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:03.350298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a40f11-85e8-437c-a8fb-707d1c7d4dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:03.352355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a40f11-85e8-437c-a8fb-707d1c7d4dd4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:03.419198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1186/2351] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:52:03 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:52:03.421123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036ff62-d963-493b-83b3-28ba1f2a5ab2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:03.433455 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036ff62-d963-493b-83b3-28ba1f2a5ab2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:03.498494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d403a0-a37d-4adb-bd2a-50b76f66ab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:03.499095 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d403a0-a37d-4adb-bd2a-50b76f66ab80 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:03.499379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d403a0-a37d-4adb-bd2a-50b76f66ab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:03.499660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d403a0-a37d-4adb-bd2a-50b76f66ab80 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:03.500375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d403a0-a37d-4adb-bd2a-50b76f66ab80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:03.502375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d403a0-a37d-4adb-bd2a-50b76f66ab80 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:03.599642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1166/2352] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:52:03 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:52:03.601984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a40f11-85e8-437c-a8fb-707d1c7d4dd4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:03.613314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a40f11-85e8-437c-a8fb-707d1c7d4dd4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:03.618179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f5f8b-9b12-48e5-b66d-e2fc67a35332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:03.618871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f5f8b-9b12-48e5-b66d-e2fc67a35332 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:03.619158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f5f8b-9b12-48e5-b66d-e2fc67a35332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:03.619435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f5f8b-9b12-48e5-b66d-e2fc67a35332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:03.620198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f5f8b-9b12-48e5-b66d-e2fc67a35332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:03.621854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f5f8b-9b12-48e5-b66d-e2fc67a35332 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:03.633119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1187/2353] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:52:03.637215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d403a0-a37d-4adb-bd2a-50b76f66ab80 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:03.647655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d403a0-a37d-4adb-bd2a-50b76f66ab80 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:03.652563 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de68cd5-77b1-47df-9c68-1e9b3db60366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:03.653242 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de68cd5-77b1-47df-9c68-1e9b3db60366 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:03.653528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de68cd5-77b1-47df-9c68-1e9b3db60366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:03.653801 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de68cd5-77b1-47df-9c68-1e9b3db60366 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:03.654571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de68cd5-77b1-47df-9c68-1e9b3db60366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:03.656168 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de68cd5-77b1-47df-9c68-1e9b3db60366 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:03.667544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1167/2354] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:52:03.671733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f5f8b-9b12-48e5-b66d-e2fc67a35332 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:03.682954 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f5f8b-9b12-48e5-b66d-e2fc67a35332 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:03.687861 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbee1c1-ec95-4b9b-9187-bcfe1bc0f99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:03.688776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbee1c1-ec95-4b9b-9187-bcfe1bc0f99d admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:03.689094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbee1c1-ec95-4b9b-9187-bcfe1bc0f99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:03.689428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbee1c1-ec95-4b9b-9187-bcfe1bc0f99d admin admin] PATH_INFO: `/v3/projects/4494b5b8c3054eeb842a6ffeb908cc45/users/9ac2092faf2b4ea09a94a8200841c652/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:03.704557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbee1c1-ec95-4b9b-9187-bcfe1bc0f99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4494b5b8c3054eeb842a6ffeb908cc45, user_id=9ac2092faf2b4ea09a94a8200841c652, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:03.707809 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbee1c1-ec95-4b9b-9187-bcfe1bc0f99d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:03.813811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1188/2355] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:52:03 2022] PUT /identity/v3/projects/4494b5b8c3054eeb842a6ffeb908cc45/users/9ac2092faf2b4ea09a94a8200841c652/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:03.824154 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff633fc-d14f-42c6-bd97-b092a1b131ac None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:03.824444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff633fc-d14f-42c6-bd97-b092a1b131ac None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:03.824728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff633fc-d14f-42c6-bd97-b092a1b131ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:03.853734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-5ff633fc-d14f-42c6-bd97-b092a1b131ac None None] MFA Rules not processed for user `9ac2092faf2b4ea09a94a8200841c652`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:03.854968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5ff633fc-d14f-42c6-bd97-b092a1b131ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:03.919137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1168/2356] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:52:03.923644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbee1c1-ec95-4b9b-9187-bcfe1bc0f99d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:03.997166 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbee1c1-ec95-4b9b-9187-bcfe1bc0f99d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:04.004434 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801493d-1c27-48fc-8683-520815548e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:04.012416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801493d-1c27-48fc-8683-520815548e04 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:04.012416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801493d-1c27-48fc-8683-520815548e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:04.012416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801493d-1c27-48fc-8683-520815548e04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:04.012416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801493d-1c27-48fc-8683-520815548e04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:04.012416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801493d-1c27-48fc-8683-520815548e04 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:04.015159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:04.114710 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff633fc-d14f-42c6-bd97-b092a1b131ac None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:04.117386 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1189/2357] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:52:03 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:52:04.133972 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff633fc-d14f-42c6-bd97-b092a1b131ac None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:04.207003 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d365ec6-816b-43b9-a8c7-9a992c351426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:04.207755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d365ec6-816b-43b9-a8c7-9a992c351426 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:04.208089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d365ec6-816b-43b9-a8c7-9a992c351426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:04.208367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d365ec6-816b-43b9-a8c7-9a992c351426 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:04.209234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d365ec6-816b-43b9-a8c7-9a992c351426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:04.211386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d365ec6-816b-43b9-a8c7-9a992c351426 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:04.221654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:04.309392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1169/2358] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:52:04 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:52:04.313219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801493d-1c27-48fc-8683-520815548e04 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:04.326526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801493d-1c27-48fc-8683-520815548e04 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:04.332665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f93d9-009a-4daf-b901-692a07b18733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:04.336592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f93d9-009a-4daf-b901-692a07b18733 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:04.336592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f93d9-009a-4daf-b901-692a07b18733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:04.336592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f93d9-009a-4daf-b901-692a07b18733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:04.336592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f93d9-009a-4daf-b901-692a07b18733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:04.337125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f93d9-009a-4daf-b901-692a07b18733 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:04.348616 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1190/2359] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:52:04.353097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d365ec6-816b-43b9-a8c7-9a992c351426 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:04.364343 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d365ec6-816b-43b9-a8c7-9a992c351426 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:04.369473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48d2a1-3355-43c8-9d4e-b1ef083d36c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:04.370367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48d2a1-3355-43c8-9d4e-b1ef083d36c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:04.370675 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48d2a1-3355-43c8-9d4e-b1ef083d36c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:04.370952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48d2a1-3355-43c8-9d4e-b1ef083d36c1 admin admin] PATH_INFO: `/v3/projects/a034cda8edf74008b0fbff11c0df491b/users/05aad1b747d645b3a82135f83ace2154/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:04.394553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48d2a1-3355-43c8-9d4e-b1ef083d36c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a034cda8edf74008b0fbff11c0df491b, user_id=05aad1b747d645b3a82135f83ace2154, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:04.399036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48d2a1-3355-43c8-9d4e-b1ef083d36c1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:04.503567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1170/2360] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:52:04 2022] PUT /identity/v3/projects/a034cda8edf74008b0fbff11c0df491b/users/05aad1b747d645b3a82135f83ace2154/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:04.507641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f93d9-009a-4daf-b901-692a07b18733 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:04.520113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f93d9-009a-4daf-b901-692a07b18733 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:04.597165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8208a0e-2a87-47f3-b2b9-b2323b3e023e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:04.597853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8208a0e-2a87-47f3-b2b9-b2323b3e023e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:04.598168 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8208a0e-2a87-47f3-b2b9-b2323b3e023e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:04.598410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8208a0e-2a87-47f3-b2b9-b2323b3e023e admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:04.599302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8208a0e-2a87-47f3-b2b9-b2323b3e023e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:04.601036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8208a0e-2a87-47f3-b2b9-b2323b3e023e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:04.614633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1191/2361] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:52:04.620762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48d2a1-3355-43c8-9d4e-b1ef083d36c1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:04.634566 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48d2a1-3355-43c8-9d4e-b1ef083d36c1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:04.639625 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0fc0c-9c1c-4c5d-b428-e498d9202093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:04.642968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0fc0c-9c1c-4c5d-b428-e498d9202093 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:04.643275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0fc0c-9c1c-4c5d-b428-e498d9202093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:04.643570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0fc0c-9c1c-4c5d-b428-e498d9202093 admin admin] PATH_INFO: `/v3/projects/a034cda8edf74008b0fbff11c0df491b/users/05aad1b747d645b3a82135f83ace2154/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:04.648606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0fc0c-9c1c-4c5d-b428-e498d9202093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a034cda8edf74008b0fbff11c0df491b, user_id=05aad1b747d645b3a82135f83ace2154, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:04.656086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0fc0c-9c1c-4c5d-b428-e498d9202093 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:04.688038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1171/2362] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:52:04 2022] PUT /identity/v3/projects/a034cda8edf74008b0fbff11c0df491b/users/05aad1b747d645b3a82135f83ace2154/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:04.700441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22c166-20b3-4212-bc83-ea47c3526817 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:04.700736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22c166-20b3-4212-bc83-ea47c3526817 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:04.701029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22c166-20b3-4212-bc83-ea47c3526817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:04.752671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4a22c166-20b3-4212-bc83-ea47c3526817 None None] MFA Rules not processed for user `05aad1b747d645b3a82135f83ace2154`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:04.754881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4a22c166-20b3-4212-bc83-ea47c3526817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:04.880148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1192/2363] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:52:04.916944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0fc0c-9c1c-4c5d-b428-e498d9202093 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:04.919090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6fb0fc0c-9c1c-4c5d-b428-e498d9202093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:05.005071 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0fc0c-9c1c-4c5d-b428-e498d9202093 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:05.008716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34494a-698e-4f1e-8f73-9075c8ae264c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:05.009534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34494a-698e-4f1e-8f73-9075c8ae264c service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:05.009904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34494a-698e-4f1e-8f73-9075c8ae264c service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:05.010180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb34494a-698e-4f1e-8f73-9075c8ae264c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:05.090996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34494a-698e-4f1e-8f73-9075c8ae264c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:05.092378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb34494a-698e-4f1e-8f73-9075c8ae264c service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:05.102431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1172/2364] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:52:07.566636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22c166-20b3-4212-bc83-ea47c3526817 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:07.577572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22c166-20b3-4212-bc83-ea47c3526817 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:07.582122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e0a3c-873d-4456-bec7-6a0912b1b6ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:07.582779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e0a3c-873d-4456-bec7-6a0912b1b6ad service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:07.583062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e0a3c-873d-4456-bec7-6a0912b1b6ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:07.594684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e0a3c-873d-4456-bec7-6a0912b1b6ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:07.644906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:07.649617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e0a3c-873d-4456-bec7-6a0912b1b6ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:07.651073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e0a3c-873d-4456-bec7-6a0912b1b6ad service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:07.663586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1193/2365] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:52:09.663909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34494a-698e-4f1e-8f73-9075c8ae264c service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:09.665434 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bb34494a-698e-4f1e-8f73-9075c8ae264c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:09.677893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb34494a-698e-4f1e-8f73-9075c8ae264c service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:09.682127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fbeac-db81-4578-89b7-829f042240bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:09.682802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fbeac-db81-4578-89b7-829f042240bc service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:09.683085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fbeac-db81-4578-89b7-829f042240bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:09.683367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fbeac-db81-4578-89b7-829f042240bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:09.693957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fbeac-db81-4578-89b7-829f042240bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:09.695426 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fbeac-db81-4578-89b7-829f042240bc service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:09.706955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1173/2366] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:52:15.536561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eeb0e-55a9-4795-a756-7fe242d67818 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:15.537401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eeb0e-55a9-4795-a756-7fe242d67818 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:15.537401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4eeb0e-55a9-4795-a756-7fe242d67818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:15.563007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-5b4eeb0e-55a9-4795-a756-7fe242d67818 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:15.564717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5b4eeb0e-55a9-4795-a756-7fe242d67818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:15.586724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fbeac-db81-4578-89b7-829f042240bc service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:15.642865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1194/2367] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:52:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:15.665909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:15.671844 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fbeac-db81-4578-89b7-829f042240bc service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:15.681306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:15.682206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:15.682500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:15.682782 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] PATH_INFO: `/v3/users/6472cd2e037d482db8c8047936132b73` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:15.684680 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6472cd2e037d482db8c8047936132b73)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:15.686703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:15.705679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eeb0e-55a9-4795-a756-7fe242d67818 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:15.707951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5b4eeb0e-55a9-4795-a756-7fe242d67818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:15.810228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4eeb0e-55a9-4795-a756-7fe242d67818 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:15.821450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbd991-cddd-485d-9927-9fafc8a85a39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:15.821450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbd991-cddd-485d-9927-9fafc8a85a39 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:15.821450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbd991-cddd-485d-9927-9fafc8a85a39 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:15.821450 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbd991-cddd-485d-9927-9fafc8a85a39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:15.832524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbd991-cddd-485d-9927-9fafc8a85a39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:15.835791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbd991-cddd-485d-9927-9fafc8a85a39 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:15.857549 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1195/2368] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:52:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:52:15.919858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6472cd2e037d482db8c8047936132b73'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:15.934150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6472cd2e037d482db8c8047936132b73'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:15.951089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6472cd2e037d482db8c8047936132b73'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:16.024927 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6472cd2e037d482db8c8047936132b73'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:16.025928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6472cd2e037d482db8c8047936132b73'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:16.041752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1174/2369] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:52:15 2022] DELETE /identity/v3/users/6472cd2e037d482db8c8047936132b73 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:52:16.042141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4752c26-f82a-4210-bbee-058f06800968 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:16.042309 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4752c26-f82a-4210-bbee-058f06800968 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:16.042591 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4752c26-f82a-4210-bbee-058f06800968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:16.066593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b4752c26-f82a-4210-bbee-058f06800968 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:16.068027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b4752c26-f82a-4210-bbee-058f06800968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:16.147528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1196/2370] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:16.160096 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:16.160096 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:16.247521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184673f7-b30c-410f-abe5-9515fdc4cad3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:16.252412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f81b1-ad30-43aa-8c02-57db26f1352c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:16.253106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f81b1-ad30-43aa-8c02-57db26f1352c service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:16.253394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f81b1-ad30-43aa-8c02-57db26f1352c service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:16.253672 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f81b1-ad30-43aa-8c02-57db26f1352c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:16.265400 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f81b1-ad30-43aa-8c02-57db26f1352c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:16.267497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f81b1-ad30-43aa-8c02-57db26f1352c service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:16.279930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1175/2371] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:52:16.788484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4752c26-f82a-4210-bbee-058f06800968 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:16.790828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b4752c26-f82a-4210-bbee-058f06800968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:16.804722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4752c26-f82a-4210-bbee-058f06800968 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:16.811081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:16.814528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:16.815070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:16.815549 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] PATH_INFO: `/v3/projects/36517f59aa7e426e90ef1ecf7cf4a811` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:16.818187 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36517f59aa7e426e90ef1ecf7cf4a811)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:16.820050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:16.849020 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:16.886894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4e78-c802-4c9f-9f2f-12998d10201c None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:16.886894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4e78-c802-4c9f-9f2f-12998d10201c None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:16.886894 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb4e78-c802-4c9f-9f2f-12998d10201c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:16.904094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-9feb4e78-c802-4c9f-9f2f-12998d10201c None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:16.905378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9feb4e78-c802-4c9f-9f2f-12998d10201c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:16.933800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36517f59aa7e426e90ef1ecf7cf4a811', 'request_id': 'req-065175dc-4130-4757-ac34-96b388ff7657'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:16.980272 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1176/2372] 10.222.0.88 () {58 vars in 967 bytes} [Mon Jul 18 16:52:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:52:16.995021 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb4e78-c802-4c9f-9f2f-12998d10201c None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:16.996443 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-9feb4e78-c802-4c9f-9f2f-12998d10201c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:17.008430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36517f59aa7e426e90ef1ecf7cf4a811', 'request_id': 'req-065175dc-4130-4757-ac34-96b388ff7657'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:17.018700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36517f59aa7e426e90ef1ecf7cf4a811', 'request_id': 'req-065175dc-4130-4757-ac34-96b388ff7657'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:17.030928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:17.037130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] The token cache is being invalidate because project 36517f59aa7e426e90ef1ecf7cf4a811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:52:17.037786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:17.039280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:17.041333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1197/2373] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:52:16 2022] DELETE /identity/v3/projects/36517f59aa7e426e90ef1ecf7cf4a811 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:17.051505 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:17.053916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:17.106990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb4e78-c802-4c9f-9f2f-12998d10201c None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:17.143065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065175dc-4130-4757-ac34-96b388ff7657 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:17.148942 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:17.151995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:17.152279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:17.152562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] PATH_INFO: `/v3/users/9ac2092faf2b4ea09a94a8200841c652` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:17.154893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac2092faf2b4ea09a94a8200841c652)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:17.159152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:17.201254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a886b-3240-4718-bc35-fc076d700914 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:17.201917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a886b-3240-4718-bc35-fc076d700914 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:17.202205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a886b-3240-4718-bc35-fc076d700914 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:17.202487 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a886b-3240-4718-bc35-fc076d700914 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:17.205753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d18a886b-3240-4718-bc35-fc076d700914 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:17.281724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a886b-3240-4718-bc35-fc076d700914 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:17.283381 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a886b-3240-4718-bc35-fc076d700914 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:17.292627 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1177/2374] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:52:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:52:17.441194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac2092faf2b4ea09a94a8200841c652'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:17.465460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac2092faf2b4ea09a94a8200841c652'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:17.472415 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac2092faf2b4ea09a94a8200841c652'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:17.601718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac2092faf2b4ea09a94a8200841c652'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:17.604643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac2092faf2b4ea09a94a8200841c652'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:17.615060 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1198/2375] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:52:17 2022] DELETE /identity/v3/users/9ac2092faf2b4ea09a94a8200841c652 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:18.075380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9916da-4c25-4a6c-ad57-aeddef8e4367 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:18.075915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9916da-4c25-4a6c-ad57-aeddef8e4367 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:18.075915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9916da-4c25-4a6c-ad57-aeddef8e4367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:18.094257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6b9916da-4c25-4a6c-ad57-aeddef8e4367 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:18.096076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6b9916da-4c25-4a6c-ad57-aeddef8e4367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:18.133826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:18.135456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:18.157113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1178/2376] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:52:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:18.170587 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9916da-4c25-4a6c-ad57-aeddef8e4367 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:18.172254 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6b9916da-4c25-4a6c-ad57-aeddef8e4367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:18.186191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:18.190475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:18.224008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae259cc-1b12-4cdf-b350-148144ded4c9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:18.231378 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:18.232268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:18.232553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:18.232832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] PATH_INFO: `/v3/projects/4494b5b8c3054eeb842a6ffeb908cc45` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:18.234145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4494b5b8c3054eeb842a6ffeb908cc45)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:18.236644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:18.249435 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9916da-4c25-4a6c-ad57-aeddef8e4367 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:18.256299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b879713-7ee3-42c7-93b3-4930e526aa0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:18.257261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b879713-7ee3-42c7-93b3-4930e526aa0d service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:18.257580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b879713-7ee3-42c7-93b3-4930e526aa0d service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:18.257863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b879713-7ee3-42c7-93b3-4930e526aa0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:18.269500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b879713-7ee3-42c7-93b3-4930e526aa0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:18.278592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b879713-7ee3-42c7-93b3-4930e526aa0d service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:18.290755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1179/2377] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:52:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:52:18.553914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4494b5b8c3054eeb842a6ffeb908cc45', 'request_id': 'req-a97f9dbb-c751-4c90-8825-28877449890b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:18.692988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4494b5b8c3054eeb842a6ffeb908cc45', 'request_id': 'req-a97f9dbb-c751-4c90-8825-28877449890b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:18.699595 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4494b5b8c3054eeb842a6ffeb908cc45', 'request_id': 'req-a97f9dbb-c751-4c90-8825-28877449890b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:18.715273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] The token cache is being invalidate because project 4494b5b8c3054eeb842a6ffeb908cc45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:52:18.720634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:18.720634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-a97f9dbb-c751-4c90-8825-28877449890b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:18.720634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1199/2378] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:52:18 2022] DELETE /identity/v3/projects/4494b5b8c3054eeb842a6ffeb908cc45 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:52:18.723268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b879713-7ee3-42c7-93b3-4930e526aa0d service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:18.734761 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b879713-7ee3-42c7-93b3-4930e526aa0d service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:18.801683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:18.802574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:18.802859 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:18.803137 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] PATH_INFO: `/v3/users/05aad1b747d645b3a82135f83ace2154` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:18.804415 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] RBAC: Authorizing `identity:delete_user(user_id=05aad1b747d645b3a82135f83ace2154)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:18.808734 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:18.974134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343e806-44b0-4089-a234-5208e0815689 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:18.974527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343e806-44b0-4089-a234-5208e0815689 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:18.974736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343e806-44b0-4089-a234-5208e0815689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:18.993133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-9343e806-44b0-4089-a234-5208e0815689 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:18.994385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9343e806-44b0-4089-a234-5208e0815689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:19.027208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05aad1b747d645b3a82135f83ace2154'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:19.034974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05aad1b747d645b3a82135f83ace2154'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:19.048106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05aad1b747d645b3a82135f83ace2154'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:19.054826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1200/2379] 10.222.0.88 () {58 vars in 967 bytes} [Mon Jul 18 16:52:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:19.067377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343e806-44b0-4089-a234-5208e0815689 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:19.068756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9343e806-44b0-4089-a234-5208e0815689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:19.133017 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05aad1b747d645b3a82135f83ace2154'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:19.133884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05aad1b747d645b3a82135f83ace2154'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:19.141112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1180/2380] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:52:18 2022] DELETE /identity/v3/users/05aad1b747d645b3a82135f83ace2154 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:19.142376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343e806-44b0-4089-a234-5208e0815689 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:19.147536 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:19.206215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee55a77-65c3-45de-88f3-5b3d28930ca1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:19.206879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee55a77-65c3-45de-88f3-5b3d28930ca1 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:19.207162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee55a77-65c3-45de-88f3-5b3d28930ca1 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:19.207451 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee55a77-65c3-45de-88f3-5b3d28930ca1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:19.209749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1ee55a77-65c3-45de-88f3-5b3d28930ca1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:19.272183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee55a77-65c3-45de-88f3-5b3d28930ca1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:19.273827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee55a77-65c3-45de-88f3-5b3d28930ca1 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:19.282460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1201/2381] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:52:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:52:19.633215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:19.634715 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:19.667107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:19.713126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c6e8b-bdb5-4763-8a95-1e0c6374f73a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:19.718211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:19.719158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:19.719445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:19.719742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] PATH_INFO: `/v3/projects/a034cda8edf74008b0fbff11c0df491b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:19.721292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a034cda8edf74008b0fbff11c0df491b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:19.723088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:19.878115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a034cda8edf74008b0fbff11c0df491b', 'request_id': 'req-ac0bff37-196a-4305-802d-2c61990c174e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:19.895572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a034cda8edf74008b0fbff11c0df491b', 'request_id': 'req-ac0bff37-196a-4305-802d-2c61990c174e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:19.902998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a034cda8edf74008b0fbff11c0df491b', 'request_id': 'req-ac0bff37-196a-4305-802d-2c61990c174e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:20.002868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] The token cache is being invalidate because project a034cda8edf74008b0fbff11c0df491b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:52:20.003049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:20.003718 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ac0bff37-196a-4305-802d-2c61990c174e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:20.005346 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1181/2382] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:52:19 2022] DELETE /identity/v3/projects/a034cda8edf74008b0fbff11c0df491b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:24.306277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee55a77-65c3-45de-88f3-5b3d28930ca1 service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:24.308049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1ee55a77-65c3-45de-88f3-5b3d28930ca1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:24.325641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee55a77-65c3-45de-88f3-5b3d28930ca1 service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:24.406665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:24.407590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:24.408385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:24.408676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] PATH_INFO: `/v3/users/cb40c15365c14eb6ae597be773bafc3a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:24.410137 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb40c15365c14eb6ae597be773bafc3a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:24.412215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:24.594114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb40c15365c14eb6ae597be773bafc3a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:24.599266 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:24.608075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb40c15365c14eb6ae597be773bafc3a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:24.616327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb40c15365c14eb6ae597be773bafc3a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:24.727094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb40c15365c14eb6ae597be773bafc3a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:24.727897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb40c15365c14eb6ae597be773bafc3a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:24.737266 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1202/2383] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:52:24 2022] DELETE /identity/v3/users/cb40c15365c14eb6ae597be773bafc3a => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:24.742877 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba9765-cf78-4e58-ab4a-833489233637 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:24.743380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba9765-cf78-4e58-ab4a-833489233637 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:24.743534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba9765-cf78-4e58-ab4a-833489233637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:24.764032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-02ba9765-cf78-4e58-ab4a-833489233637 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:24.765419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-02ba9765-cf78-4e58-ab4a-833489233637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:24.833238 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1182/2384] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:24.847198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:24.848820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:24.912465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecf7a3-cc86-42ef-8cab-a17321997763 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:24.916359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdcd28-60f7-47a5-a97f-e42f6d924e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:24.917074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdcd28-60f7-47a5-a97f-e42f6d924e65 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:24.917877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdcd28-60f7-47a5-a97f-e42f6d924e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:24.917877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdcd28-60f7-47a5-a97f-e42f6d924e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:24.933711 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdcd28-60f7-47a5-a97f-e42f6d924e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:24.935418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdcd28-60f7-47a5-a97f-e42f6d924e65 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:24.950062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1203/2385] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:52:25.254951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba9765-cf78-4e58-ab4a-833489233637 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:25.256443 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-02ba9765-cf78-4e58-ab4a-833489233637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:25.268866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba9765-cf78-4e58-ab4a-833489233637 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:25.273516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:25.274363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:25.274648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:25.274919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] PATH_INFO: `/v3/projects/b4ac8a7f26f6423fb63683053905a4fe` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:25.276189 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4ac8a7f26f6423fb63683053905a4fe)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:25.278484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:25.430143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:25.432818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4ac8a7f26f6423fb63683053905a4fe', 'request_id': 'req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:25.439421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4ac8a7f26f6423fb63683053905a4fe', 'request_id': 'req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:25.445292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4ac8a7f26f6423fb63683053905a4fe', 'request_id': 'req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:25.641128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] The token cache is being invalidate because project b4ac8a7f26f6423fb63683053905a4fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:52:25.641425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:25.642076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bcffb067-7afd-4f84-97bf-c638e7a4cdd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:25.644332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1183/2386] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:52:25 2022] DELETE /identity/v3/projects/b4ac8a7f26f6423fb63683053905a4fe => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:25.874177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdcd28-60f7-47a5-a97f-e42f6d924e65 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:25.875723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-30bdcd28-60f7-47a5-a97f-e42f6d924e65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:25.889238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdcd28-60f7-47a5-a97f-e42f6d924e65 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:25.953648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:25.954013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:25.954289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:25.954560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] PATH_INFO: `/v3/users/f14600f242a8490ca4d83b6c6ad2ab47` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:25.956311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f14600f242a8490ca4d83b6c6ad2ab47)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:25.958251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:26.174821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f14600f242a8490ca4d83b6c6ad2ab47'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:26.186721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f14600f242a8490ca4d83b6c6ad2ab47'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:26.195478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f14600f242a8490ca4d83b6c6ad2ab47'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:26.197461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:26.316218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f14600f242a8490ca4d83b6c6ad2ab47'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:26.316759 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f14600f242a8490ca4d83b6c6ad2ab47'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:26.323300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1204/2387] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:52:25 2022] DELETE /identity/v3/users/f14600f242a8490ca4d83b6c6ad2ab47 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:26.328168 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cff307-e37c-4a54-9d95-20e74ffebf24 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:26.328695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cff307-e37c-4a54-9d95-20e74ffebf24 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:26.328793 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cff307-e37c-4a54-9d95-20e74ffebf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:26.350082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-35cff307-e37c-4a54-9d95-20e74ffebf24 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:26.351302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-35cff307-e37c-4a54-9d95-20e74ffebf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:26.419337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1184/2388] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:26.430588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:26.432053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:26.497172 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb29a6-3803-408d-ac7f-146caf67991a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:26.501052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd4ea8-2c84-4211-8ca4-bd31de043a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:26.502333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd4ea8-2c84-4211-8ca4-bd31de043a4c service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:26.502580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd4ea8-2c84-4211-8ca4-bd31de043a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:26.502904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd4ea8-2c84-4211-8ca4-bd31de043a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:26.513509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd4ea8-2c84-4211-8ca4-bd31de043a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:26.515169 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd4ea8-2c84-4211-8ca4-bd31de043a4c service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:26.526408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1205/2389] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:52:27.053819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cff307-e37c-4a54-9d95-20e74ffebf24 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:27.055750 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-35cff307-e37c-4a54-9d95-20e74ffebf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:27.076052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cff307-e37c-4a54-9d95-20e74ffebf24 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:27.087369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:27.087369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:27.087369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:27.087369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] PATH_INFO: `/v3/projects/a2edfc0ea1824972bc4ceacb03da421c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:27.087970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2edfc0ea1824972bc4ceacb03da421c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:27.089377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:27.219267 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2edfc0ea1824972bc4ceacb03da421c', 'request_id': 'req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:27.227751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2edfc0ea1824972bc4ceacb03da421c', 'request_id': 'req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:27.234334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2edfc0ea1824972bc4ceacb03da421c', 'request_id': 'req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:27.327883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] The token cache is being invalidate because project a2edfc0ea1824972bc4ceacb03da421c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:52:27.328514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:27.329355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ad6b3b81-7358-42f1-b021-9d27a6fc2d91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:27.334673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1185/2390] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:52:27 2022] DELETE /identity/v3/projects/a2edfc0ea1824972bc4ceacb03da421c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:27.337874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b3ae2-81ea-419d-905c-716862e66516 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:27.338399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b3ae2-81ea-419d-905c-716862e66516 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:27.338526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b3ae2-81ea-419d-905c-716862e66516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:27.358567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-ed7b3ae2-81ea-419d-905c-716862e66516 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:27.359712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ed7b3ae2-81ea-419d-905c-716862e66516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:27.445158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1206/2391] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:27.455907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce64c5-fd0f-4743-a5da-865df862681b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:27.456212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce64c5-fd0f-4743-a5da-865df862681b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:27.456528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce64c5-fd0f-4743-a5da-865df862681b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:27.481857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-bcce64c5-fd0f-4743-a5da-865df862681b None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:27.483863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bcce64c5-fd0f-4743-a5da-865df862681b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:27.492673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1186/2392] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:27.498260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b3ae2-81ea-419d-905c-716862e66516 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:27.510037 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b3ae2-81ea-419d-905c-716862e66516 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:27.515286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b1d38-8b70-47bd-adae-98a85a199ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:27.515959 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b1d38-8b70-47bd-adae-98a85a199ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:27.516253 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b1d38-8b70-47bd-adae-98a85a199ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:27.516529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b1d38-8b70-47bd-adae-98a85a199ea2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:27.517900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b1d38-8b70-47bd-adae-98a85a199ea2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:52:27.518210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b1d38-8b70-47bd-adae-98a85a199ea2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:27.520238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b1d38-8b70-47bd-adae-98a85a199ea2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:27.537724 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1207/2393] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:52:27.542998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce64c5-fd0f-4743-a5da-865df862681b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:27.555723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce64c5-fd0f-4743-a5da-865df862681b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:27.561292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606f4f1-fbf5-4081-a338-359c45134df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:27.561998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606f4f1-fbf5-4081-a338-359c45134df1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:27.562305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606f4f1-fbf5-4081-a338-359c45134df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:27.563555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606f4f1-fbf5-4081-a338-359c45134df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:27.563555 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606f4f1-fbf5-4081-a338-359c45134df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:27.566125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606f4f1-fbf5-4081-a338-359c45134df1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:27.676893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1187/2394] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:52:27 2022] POST /identity/v3/projects => generated 369 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:52:27.681473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b1d38-8b70-47bd-adae-98a85a199ea2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:27.692436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b1d38-8b70-47bd-adae-98a85a199ea2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:27.757894 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58665989-ffdf-40a4-867b-08503a52ae1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:27.758682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58665989-ffdf-40a4-867b-08503a52ae1e admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:27.758975 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58665989-ffdf-40a4-867b-08503a52ae1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:27.759317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58665989-ffdf-40a4-867b-08503a52ae1e admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:27.760333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58665989-ffdf-40a4-867b-08503a52ae1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:27.762994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58665989-ffdf-40a4-867b-08503a52ae1e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:27.854681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1208/2395] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:52:27 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:52:27.859166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606f4f1-fbf5-4081-a338-359c45134df1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:27.872849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606f4f1-fbf5-4081-a338-359c45134df1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:27.880765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28540761-4b46-4187-ac73-82758fc9d186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:27.881732 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28540761-4b46-4187-ac73-82758fc9d186 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:27.882387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28540761-4b46-4187-ac73-82758fc9d186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:27.882885 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28540761-4b46-4187-ac73-82758fc9d186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:27.883842 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28540761-4b46-4187-ac73-82758fc9d186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:27.886115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28540761-4b46-4187-ac73-82758fc9d186 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:27.915779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1188/2396] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:52:27.921325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58665989-ffdf-40a4-867b-08503a52ae1e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:27.933576 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58665989-ffdf-40a4-867b-08503a52ae1e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:27.943394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786c835-1b0e-4d42-8635-cf244323648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:27.944088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786c835-1b0e-4d42-8635-cf244323648a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:27.944397 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786c835-1b0e-4d42-8635-cf244323648a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:27.944664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786c835-1b0e-4d42-8635-cf244323648a admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:27.947594 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786c835-1b0e-4d42-8635-cf244323648a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:27.949269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786c835-1b0e-4d42-8635-cf244323648a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:27.957972 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:27.967317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1209/2397] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:52:27.972033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28540761-4b46-4187-ac73-82758fc9d186 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:27.984401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28540761-4b46-4187-ac73-82758fc9d186 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:27.989842 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278398f-d83c-4766-befb-387fd7097ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:27.993614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278398f-d83c-4766-befb-387fd7097ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:27.993614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278398f-d83c-4766-befb-387fd7097ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:27.993614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278398f-d83c-4766-befb-387fd7097ccb admin admin] PATH_INFO: `/v3/projects/a29e46896ebd4b739acb25872ed5ab2c/users/7a2578d5eb784a56819aa1b176a12b08/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:28.014736 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278398f-d83c-4766-befb-387fd7097ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29e46896ebd4b739acb25872ed5ab2c, user_id=7a2578d5eb784a56819aa1b176a12b08, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:28.017625 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278398f-d83c-4766-befb-387fd7097ccb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:28.021768 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:28.156441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1189/2398] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:52:27 2022] PUT /identity/v3/projects/a29e46896ebd4b739acb25872ed5ab2c/users/7a2578d5eb784a56819aa1b176a12b08/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:28.168179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5d749-809a-440c-b716-8c9e4067fb71 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:28.168538 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5d749-809a-440c-b716-8c9e4067fb71 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:28.174096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5d749-809a-440c-b716-8c9e4067fb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:28.205404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c5d5d749-809a-440c-b716-8c9e4067fb71 None None] MFA Rules not processed for user `7a2578d5eb784a56819aa1b176a12b08`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:28.206841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c5d5d749-809a-440c-b716-8c9e4067fb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:28.281662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1210/2399] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:28.302178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278398f-d83c-4766-befb-387fd7097ccb admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:28.305227 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e278398f-d83c-4766-befb-387fd7097ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:28.382574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278398f-d83c-4766-befb-387fd7097ccb admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:28.386673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeca656-65ad-4611-8f56-ed03f0d71b50 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54c7869983bf4534b3e730af0185eac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:28.395181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeca656-65ad-4611-8f56-ed03f0d71b50 service glance] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:28.395181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeca656-65ad-4611-8f56-ed03f0d71b50 service glance] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:28.395181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeca656-65ad-4611-8f56-ed03f0d71b50 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:28.399047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeca656-65ad-4611-8f56-ed03f0d71b50 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:28.400551 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeca656-65ad-4611-8f56-ed03f0d71b50 service glance] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:28.412378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1190/2400] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:52:38.278148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=32 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:38.413349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:54.643783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5d749-809a-440c-b716-8c9e4067fb71 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:54.646124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c5d5d749-809a-440c-b716-8c9e4067fb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:54.724389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5d749-809a-440c-b716-8c9e4067fb71 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:54.729340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:54.730224 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:54.730507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:54.730777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] PATH_INFO: `/v3/users/c14b337eed0e4877bf6dcba04724f9f5` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:54.732149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14b337eed0e4877bf6dcba04724f9f5)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:54.734158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:54.917908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14b337eed0e4877bf6dcba04724f9f5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:54.931214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14b337eed0e4877bf6dcba04724f9f5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:54.939354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14b337eed0e4877bf6dcba04724f9f5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:55.018137 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14b337eed0e4877bf6dcba04724f9f5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:55.018754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14b337eed0e4877bf6dcba04724f9f5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:55.025452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1211/2401] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:52:54 2022] DELETE /identity/v3/users/c14b337eed0e4877bf6dcba04724f9f5 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:55.030271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13a4f1-a1a8-483d-a3c3-7cecc335b9f2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:55.030777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13a4f1-a1a8-483d-a3c3-7cecc335b9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:55.030868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13a4f1-a1a8-483d-a3c3-7cecc335b9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:55.055658 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3c13a4f1-a1a8-483d-a3c3-7cecc335b9f2 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:55.056419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3c13a4f1-a1a8-483d-a3c3-7cecc335b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:55.123473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1191/2402] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:55.137703 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:55.137703 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:55.201930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63a6a8-d58d-4dc7-8ed6-46a9224c1ac0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:55.206914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce773e-50b2-4fa3-9d61-bb1eda21d5bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:55.207036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce773e-50b2-4fa3-9d61-bb1eda21d5bc service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:55.207315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce773e-50b2-4fa3-9d61-bb1eda21d5bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:55.207592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce773e-50b2-4fa3-9d61-bb1eda21d5bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:55.218580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce773e-50b2-4fa3-9d61-bb1eda21d5bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:55.220067 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce773e-50b2-4fa3-9d61-bb1eda21d5bc service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:55.232922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1212/2403] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:52:55.671160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13a4f1-a1a8-483d-a3c3-7cecc335b9f2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:55.674806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3c13a4f1-a1a8-483d-a3c3-7cecc335b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:55.702211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13a4f1-a1a8-483d-a3c3-7cecc335b9f2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:55.709271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:55.710380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:55.710709 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:55.712017 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] PATH_INFO: `/v3/projects/a9024588293448c3ac92cc12a5f24d76` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:55.715062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9024588293448c3ac92cc12a5f24d76)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:55.718185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:55.863485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9024588293448c3ac92cc12a5f24d76', 'request_id': 'req-abff9626-26ed-4e12-b822-5184108d0bb6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:55.869758 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9024588293448c3ac92cc12a5f24d76', 'request_id': 'req-abff9626-26ed-4e12-b822-5184108d0bb6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:55.875321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9024588293448c3ac92cc12a5f24d76', 'request_id': 'req-abff9626-26ed-4e12-b822-5184108d0bb6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:55.968655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] The token cache is being invalidate because project a9024588293448c3ac92cc12a5f24d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:52:55.968655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:55.968655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:55.968655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1192/2404] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:52:55 2022] DELETE /identity/v3/projects/a9024588293448c3ac92cc12a5f24d76 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:55.972449 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce773e-50b2-4fa3-9d61-bb1eda21d5bc service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:55.982939 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce773e-50b2-4fa3-9d61-bb1eda21d5bc service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:56.002286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:56.041945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba596054-db23-49d1-8040-f7c2e4f354ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:56.042819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba596054-db23-49d1-8040-f7c2e4f354ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:56.043128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba596054-db23-49d1-8040-f7c2e4f354ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:56.043404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba596054-db23-49d1-8040-f7c2e4f354ab admin admin] PATH_INFO: `/v3/users/a60b2e1c43c04b698a8fd1e1712f52ef` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:56.045056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba596054-db23-49d1-8040-f7c2e4f354ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a60b2e1c43c04b698a8fd1e1712f52ef)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:56.051202 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba596054-db23-49d1-8040-f7c2e4f354ab admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:56.244351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ba596054-db23-49d1-8040-f7c2e4f354ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a60b2e1c43c04b698a8fd1e1712f52ef'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:56.273060 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ba596054-db23-49d1-8040-f7c2e4f354ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a60b2e1c43c04b698a8fd1e1712f52ef'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:56.282817 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ba596054-db23-49d1-8040-f7c2e4f354ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a60b2e1c43c04b698a8fd1e1712f52ef'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:56.399240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ba596054-db23-49d1-8040-f7c2e4f354ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a60b2e1c43c04b698a8fd1e1712f52ef'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:56.400024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ba596054-db23-49d1-8040-f7c2e4f354ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a60b2e1c43c04b698a8fd1e1712f52ef'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:56.408553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1213/2405] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:52:55 2022] DELETE /identity/v3/users/a60b2e1c43c04b698a8fd1e1712f52ef => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:56.844524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:56.845512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:56.868869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff9626-26ed-4e12-b822-5184108d0bb6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:56.933835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:56.934745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:56.935030 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:56.935306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] PATH_INFO: `/v3/projects/003dde174e54459bbd881966bf2577fa` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:56.937784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] RBAC: Authorizing `identity:delete_project(project_id=003dde174e54459bbd881966bf2577fa)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:56.939990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:57.025357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:57.042779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '003dde174e54459bbd881966bf2577fa', 'request_id': 'req-a014885a-4a65-4786-9bc3-548de875302b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:57.049302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '003dde174e54459bbd881966bf2577fa', 'request_id': 'req-a014885a-4a65-4786-9bc3-548de875302b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:57.055180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '003dde174e54459bbd881966bf2577fa', 'request_id': 'req-a014885a-4a65-4786-9bc3-548de875302b'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:57.166154 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] The token cache is being invalidate because project 003dde174e54459bbd881966bf2577fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:52:57.166451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:57.167099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a014885a-4a65-4786-9bc3-548de875302b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:52:57.168713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1193/2406] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:52:56 2022] DELETE /identity/v3/projects/003dde174e54459bbd881966bf2577fa => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:57.173792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3ba2d-31bc-494f-99df-c65faf909a1a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:57.174282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3ba2d-31bc-494f-99df-c65faf909a1a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:57.174381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3ba2d-31bc-494f-99df-c65faf909a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:57.194965 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-eec3ba2d-31bc-494f-99df-c65faf909a1a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:57.197106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eec3ba2d-31bc-494f-99df-c65faf909a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:57.262180 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1214/2407] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:57.270546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc52d-15bf-42ca-852f-b96cbf670ecf None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:57.270773 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc52d-15bf-42ca-852f-b96cbf670ecf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:57.271085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bc52d-15bf-42ca-852f-b96cbf670ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:57.290723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4a2bc52d-15bf-42ca-852f-b96cbf670ecf None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:57.291965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4a2bc52d-15bf-42ca-852f-b96cbf670ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:57.306540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1194/2408] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:52:57.312221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3ba2d-31bc-494f-99df-c65faf909a1a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:57.330489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3ba2d-31bc-494f-99df-c65faf909a1a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:57.336853 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a6fab-bebb-4956-958d-a491b83ac19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:57.337784 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a6fab-bebb-4956-958d-a491b83ac19e admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:57.338062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a6fab-bebb-4956-958d-a491b83ac19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:57.339788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a6fab-bebb-4956-958d-a491b83ac19e admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:57.340937 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a6fab-bebb-4956-958d-a491b83ac19e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:52:57.341314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a6fab-bebb-4956-958d-a491b83ac19e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:57.343476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a6fab-bebb-4956-958d-a491b83ac19e admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:57.357708 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1215/2409] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:52:57.362524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bc52d-15bf-42ca-852f-b96cbf670ecf None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:57.373306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bc52d-15bf-42ca-852f-b96cbf670ecf None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:57.378771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603424e6-75ea-47e9-9cd9-fd213aa57863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:57.379448 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603424e6-75ea-47e9-9cd9-fd213aa57863 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:57.379738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603424e6-75ea-47e9-9cd9-fd213aa57863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:57.380068 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603424e6-75ea-47e9-9cd9-fd213aa57863 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:57.380745 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603424e6-75ea-47e9-9cd9-fd213aa57863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:57.383739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603424e6-75ea-47e9-9cd9-fd213aa57863 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:57.538290 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1195/2410] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:52:57 2022] POST /identity/v3/projects => generated 357 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:52:57.539532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a6fab-bebb-4956-958d-a491b83ac19e admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:57.551386 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a6fab-bebb-4956-958d-a491b83ac19e admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:57.581866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:57.615415 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897888a-d9cb-45b5-ad9f-b17f7abe6b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:57.616095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897888a-d9cb-45b5-ad9f-b17f7abe6b83 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:57.617390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897888a-d9cb-45b5-ad9f-b17f7abe6b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:57.617390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897888a-d9cb-45b5-ad9f-b17f7abe6b83 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:57.617390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897888a-d9cb-45b5-ad9f-b17f7abe6b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:57.619265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897888a-d9cb-45b5-ad9f-b17f7abe6b83 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:57.696267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1216/2411] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:52:57 2022] POST /identity/v3/users => generated 279 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:52:57.700420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603424e6-75ea-47e9-9cd9-fd213aa57863 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:57.711912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603424e6-75ea-47e9-9cd9-fd213aa57863 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:57.716603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9c767-79a3-4f30-9c6e-135cce0eb726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:57.717295 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9c767-79a3-4f30-9c6e-135cce0eb726 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:57.717581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9c767-79a3-4f30-9c6e-135cce0eb726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:57.717858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9c767-79a3-4f30-9c6e-135cce0eb726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:57.718591 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9c767-79a3-4f30-9c6e-135cce0eb726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:57.720198 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9c767-79a3-4f30-9c6e-135cce0eb726 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:57.731208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1196/2412] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:52:57.735879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897888a-d9cb-45b5-ad9f-b17f7abe6b83 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:57.745881 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897888a-d9cb-45b5-ad9f-b17f7abe6b83 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:57.751351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5c415-989c-4825-8816-f44990017fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:57.752016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5c415-989c-4825-8816-f44990017fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:57.752303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5c415-989c-4825-8816-f44990017fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:57.752579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5c415-989c-4825-8816-f44990017fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:57.753440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5c415-989c-4825-8816-f44990017fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:57.755044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5c415-989c-4825-8816-f44990017fd8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:57.765582 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1217/2413] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:52:57.769600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9c767-79a3-4f30-9c6e-135cce0eb726 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:57.780230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9c767-79a3-4f30-9c6e-135cce0eb726 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:57.786256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd7bde-010f-4da6-8988-e6602e118b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:57.787214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd7bde-010f-4da6-8988-e6602e118b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:57.787608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd7bde-010f-4da6-8988-e6602e118b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:57.787883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd7bde-010f-4da6-8988-e6602e118b69 admin admin] PATH_INFO: `/v3/projects/90e84f440a9241bb87fc5732a9553a57/users/988755104aea4d3cb784ff0c397e75fd/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:57.805150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd7bde-010f-4da6-8988-e6602e118b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90e84f440a9241bb87fc5732a9553a57, user_id=988755104aea4d3cb784ff0c397e75fd, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:57.809317 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd7bde-010f-4da6-8988-e6602e118b69 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:57.887184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1197/2414] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:52:57 2022] PUT /identity/v3/projects/90e84f440a9241bb87fc5732a9553a57/users/988755104aea4d3cb784ff0c397e75fd/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:57.898387 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c9b1d-ff20-4867-8ae8-e40c76f8ad23 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:57.898823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c9b1d-ff20-4867-8ae8-e40c76f8ad23 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:57.899178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c9b1d-ff20-4867-8ae8-e40c76f8ad23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:57.931267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-cd2c9b1d-ff20-4867-8ae8-e40c76f8ad23 None None] MFA Rules not processed for user `988755104aea4d3cb784ff0c397e75fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:57.932732 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cd2c9b1d-ff20-4867-8ae8-e40c76f8ad23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:58.002170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1218/2415] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:57 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:58.007607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd7bde-010f-4da6-8988-e6602e118b69 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:58.074118 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd7bde-010f-4da6-8988-e6602e118b69 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:58.079363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d296da-b328-4b88-8e5b-4a4ac0f02587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:58.080034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d296da-b328-4b88-8e5b-4a4ac0f02587 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:58.080332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d296da-b328-4b88-8e5b-4a4ac0f02587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:58.080614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d296da-b328-4b88-8e5b-4a4ac0f02587 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:58.081379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d296da-b328-4b88-8e5b-4a4ac0f02587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:58.083650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d296da-b328-4b88-8e5b-4a4ac0f02587 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:58.180560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1198/2416] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:52:58 2022] POST /identity/v3/projects => generated 357 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:52:58.183930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c9b1d-ff20-4867-8ae8-e40c76f8ad23 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:58.194956 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c9b1d-ff20-4867-8ae8-e40c76f8ad23 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:58.253634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7e9bc-2966-4323-b581-ebaa7e95c288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:58.254311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7e9bc-2966-4323-b581-ebaa7e95c288 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:58.254597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7e9bc-2966-4323-b581-ebaa7e95c288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:58.254873 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7e9bc-2966-4323-b581-ebaa7e95c288 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:58.255575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7e9bc-2966-4323-b581-ebaa7e95c288 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:58.257677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7e9bc-2966-4323-b581-ebaa7e95c288 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:58.364035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1219/2417] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:52:58 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:52:58.365407 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d296da-b328-4b88-8e5b-4a4ac0f02587 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:58.376677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d296da-b328-4b88-8e5b-4a4ac0f02587 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:58.381583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55985b4f-5c13-43e5-8f7f-0419b4946e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:58.382321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55985b4f-5c13-43e5-8f7f-0419b4946e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:58.382606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55985b4f-5c13-43e5-8f7f-0419b4946e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:58.382943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55985b4f-5c13-43e5-8f7f-0419b4946e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:58.383689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55985b4f-5c13-43e5-8f7f-0419b4946e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:58.385513 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55985b4f-5c13-43e5-8f7f-0419b4946e17 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:58.402049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1199/2418] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:52:58.405475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7e9bc-2966-4323-b581-ebaa7e95c288 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:58.416332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7e9bc-2966-4323-b581-ebaa7e95c288 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:58.425047 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b026cf-7eef-4dca-bb7d-7970dfba38cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:58.426036 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b026cf-7eef-4dca-bb7d-7970dfba38cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:58.426405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b026cf-7eef-4dca-bb7d-7970dfba38cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:58.426716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b026cf-7eef-4dca-bb7d-7970dfba38cd admin admin] PATH_INFO: `/v3/projects/86638b478b024df29c0cb8e4cbf583c7/users/d95169a9c138445baa7686246e0a9abc/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:58.443352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b026cf-7eef-4dca-bb7d-7970dfba38cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=86638b478b024df29c0cb8e4cbf583c7, user_id=d95169a9c138445baa7686246e0a9abc, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:58.446075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b026cf-7eef-4dca-bb7d-7970dfba38cd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:58.531886 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1220/2419] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:52:58 2022] PUT /identity/v3/projects/86638b478b024df29c0cb8e4cbf583c7/users/d95169a9c138445baa7686246e0a9abc/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:58.536746 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55985b4f-5c13-43e5-8f7f-0419b4946e17 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:58.546742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55985b4f-5c13-43e5-8f7f-0419b4946e17 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:58.569174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:58.608807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d96de5-eb39-4bf4-bb38-b9bee6a35a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:58.609963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d96de5-eb39-4bf4-bb38-b9bee6a35a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:58.610248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d96de5-eb39-4bf4-bb38-b9bee6a35a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:58.610526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d96de5-eb39-4bf4-bb38-b9bee6a35a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:58.611257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d96de5-eb39-4bf4-bb38-b9bee6a35a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:58.612869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d96de5-eb39-4bf4-bb38-b9bee6a35a06 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:58.622699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1200/2420] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:52:58.627845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b026cf-7eef-4dca-bb7d-7970dfba38cd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:58.633303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:52:58.638426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b026cf-7eef-4dca-bb7d-7970dfba38cd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:58.643091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f14686-4a3a-4931-ad9d-5f3867d543b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:58.643977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f14686-4a3a-4931-ad9d-5f3867d543b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:58.644258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f14686-4a3a-4931-ad9d-5f3867d543b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:58.644532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f14686-4a3a-4931-ad9d-5f3867d543b2 admin admin] PATH_INFO: `/v3/projects/86638b478b024df29c0cb8e4cbf583c7/users/d95169a9c138445baa7686246e0a9abc/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:58.646577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f14686-4a3a-4931-ad9d-5f3867d543b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86638b478b024df29c0cb8e4cbf583c7, user_id=d95169a9c138445baa7686246e0a9abc, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:58.650279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f14686-4a3a-4931-ad9d-5f3867d543b2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:58.660029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1221/2421] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:52:58 2022] PUT /identity/v3/projects/86638b478b024df29c0cb8e4cbf583c7/users/d95169a9c138445baa7686246e0a9abc/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:52:58.670226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b5ee0-3bf0-417b-88cd-81ea6a462c1d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:58.670525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b5ee0-3bf0-417b-88cd-81ea6a462c1d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:58.670812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b5ee0-3bf0-417b-88cd-81ea6a462c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:58.704917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b13b5ee0-3bf0-417b-88cd-81ea6a462c1d None None] MFA Rules not processed for user `d95169a9c138445baa7686246e0a9abc`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:52:58.706253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b13b5ee0-3bf0-417b-88cd-81ea6a462c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:58.773617 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1201/2422] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:52:58 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:52:58.785893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f14686-4a3a-4931-ad9d-5f3867d543b2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:58.787620 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e5f14686-4a3a-4931-ad9d-5f3867d543b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:58.869832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f14686-4a3a-4931-ad9d-5f3867d543b2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:58.875672 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0545997-717d-4a3a-b9f7-08df5823920f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:58.876475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0545997-717d-4a3a-b9f7-08df5823920f service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:58.876770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0545997-717d-4a3a-b9f7-08df5823920f service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:58.877085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0545997-717d-4a3a-b9f7-08df5823920f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:58.886963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0545997-717d-4a3a-b9f7-08df5823920f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:58.888433 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0545997-717d-4a3a-b9f7-08df5823920f service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:58.902207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1222/2423] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:58 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:52:59.197481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b5ee0-3bf0-417b-88cd-81ea6a462c1d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:59.198780 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b13b5ee0-3bf0-417b-88cd-81ea6a462c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:59.212248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b5ee0-3bf0-417b-88cd-81ea6a462c1d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:59.224260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a548f33-23b4-442e-a2c3-52b8ddd70116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:59.226621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a548f33-23b4-442e-a2c3-52b8ddd70116 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:59.226621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a548f33-23b4-442e-a2c3-52b8ddd70116 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:59.226621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a548f33-23b4-442e-a2c3-52b8ddd70116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:59.287164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a548f33-23b4-442e-a2c3-52b8ddd70116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:59.288645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a548f33-23b4-442e-a2c3-52b8ddd70116 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:59.298688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1202/2424] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:59 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:52:59.535823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0545997-717d-4a3a-b9f7-08df5823920f service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:52:59.537349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e0545997-717d-4a3a-b9f7-08df5823920f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:59.599473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0545997-717d-4a3a-b9f7-08df5823920f service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:52:59.603491 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957f630-95a4-434d-bc5c-f89b2371562b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:52:59.604183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957f630-95a4-434d-bc5c-f89b2371562b service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:52:59.604524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957f630-95a4-434d-bc5c-f89b2371562b service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:52:59.604800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957f630-95a4-434d-bc5c-f89b2371562b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:52:59.606634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-7957f630-95a4-434d-bc5c-f89b2371562b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:52:59.672779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957f630-95a4-434d-bc5c-f89b2371562b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:52:59.674431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957f630-95a4-434d-bc5c-f89b2371562b service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:52:59.684462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1223/2425] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:52:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:01.210464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a548f33-23b4-442e-a2c3-52b8ddd70116 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:01.212129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5a548f33-23b4-442e-a2c3-52b8ddd70116 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:01.261213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:01.281032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a548f33-23b4-442e-a2c3-52b8ddd70116 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:01.285765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:01.286645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:01.286925 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:01.287202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] PATH_INFO: `/v3/users/4ab04c8cb3f14a359ffebe8ade37dd74` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:01.288409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab04c8cb3f14a359ffebe8ade37dd74)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:01.296930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:01.489323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab04c8cb3f14a359ffebe8ade37dd74'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:01.499645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab04c8cb3f14a359ffebe8ade37dd74'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:01.514679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab04c8cb3f14a359ffebe8ade37dd74'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:01.611807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab04c8cb3f14a359ffebe8ade37dd74'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:01.612472 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab04c8cb3f14a359ffebe8ade37dd74'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:01.619066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1203/2426] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:53:01 2022] DELETE /identity/v3/users/4ab04c8cb3f14a359ffebe8ade37dd74 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:01.623585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718873-9e61-4fd8-86c2-80e3715f10fb None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:01.624043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718873-9e61-4fd8-86c2-80e3715f10fb None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:01.624157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718873-9e61-4fd8-86c2-80e3715f10fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:01.642237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-dc718873-9e61-4fd8-86c2-80e3715f10fb None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:01.643363 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dc718873-9e61-4fd8-86c2-80e3715f10fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:01.683149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:01.705806 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1224/2427] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:01.717656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:01.720150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:01.791728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb44c8-a95b-4194-ab75-7b839805b0d0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:01.796066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092c724-e054-4e6e-a30e-32e9deb58eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:01.796794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c724-e054-4e6e-a30e-32e9deb58eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:01.797102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c724-e054-4e6e-a30e-32e9deb58eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:01.797386 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092c724-e054-4e6e-a30e-32e9deb58eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:01.809445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092c724-e054-4e6e-a30e-32e9deb58eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:01.810520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092c724-e054-4e6e-a30e-32e9deb58eb9 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:01.823420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1204/2428] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:02.590112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc718873-9e61-4fd8-86c2-80e3715f10fb None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:02.592163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dc718873-9e61-4fd8-86c2-80e3715f10fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:02.608785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc718873-9e61-4fd8-86c2-80e3715f10fb None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:02.621873 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:02.623145 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:02.623860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:02.625244 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] PATH_INFO: `/v3/projects/fce867e89c2e456996efceb5897f397f` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:02.629280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fce867e89c2e456996efceb5897f397f)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:02.629280 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:02.834466 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fce867e89c2e456996efceb5897f397f', 'request_id': 'req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:02.892191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fce867e89c2e456996efceb5897f397f', 'request_id': 'req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:02.902119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fce867e89c2e456996efceb5897f397f', 'request_id': 'req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:02.915535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] The token cache is being invalidate because project fce867e89c2e456996efceb5897f397f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:53:02.915873 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:02.916570 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5acc9e9e-bf24-481b-beb0-ac8c979aa69b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:02.918231 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1225/2429] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:53:02 2022] DELETE /identity/v3/projects/fce867e89c2e456996efceb5897f397f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:53:02.923928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aed42c-831c-40a2-bd94-495489a472b1 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:02.924437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aed42c-831c-40a2-bd94-495489a472b1 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:02.926220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aed42c-831c-40a2-bd94-495489a472b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:02.957377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e0aed42c-831c-40a2-bd94-495489a472b1 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:02.959303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e0aed42c-831c-40a2-bd94-495489a472b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:03.036134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1205/2430] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:03.044289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae08ef-9d33-4b91-a8fe-b8992b409f93 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:03.044626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae08ef-9d33-4b91-a8fe-b8992b409f93 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:03.045128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae08ef-9d33-4b91-a8fe-b8992b409f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:03.068704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e5ae08ef-9d33-4b91-a8fe-b8992b409f93 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:03.070543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e5ae08ef-9d33-4b91-a8fe-b8992b409f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:03.080943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1226/2431] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:03.084641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aed42c-831c-40a2-bd94-495489a472b1 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:03.103483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aed42c-831c-40a2-bd94-495489a472b1 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:03.110235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752eb84-60c4-4efc-bf5e-91259b238e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:03.111126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752eb84-60c4-4efc-bf5e-91259b238e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:03.111476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752eb84-60c4-4efc-bf5e-91259b238e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:03.119485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752eb84-60c4-4efc-bf5e-91259b238e22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:03.119485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752eb84-60c4-4efc-bf5e-91259b238e22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:53:03.119485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752eb84-60c4-4efc-bf5e-91259b238e22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:03.119485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752eb84-60c4-4efc-bf5e-91259b238e22 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:03.119485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:03.131494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1206/2432] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:53:03.135766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae08ef-9d33-4b91-a8fe-b8992b409f93 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:03.146808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae08ef-9d33-4b91-a8fe-b8992b409f93 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:03.153693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a1bba-7207-409a-899e-56fbe08a4bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:03.154672 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a1bba-7207-409a-899e-56fbe08a4bfa admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:03.154998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a1bba-7207-409a-899e-56fbe08a4bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:03.155232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a1bba-7207-409a-899e-56fbe08a4bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:03.156046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a1bba-7207-409a-899e-56fbe08a4bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:03.158066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a1bba-7207-409a-899e-56fbe08a4bfa admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:03.321152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1227/2433] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:53:03 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:03.325326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752eb84-60c4-4efc-bf5e-91259b238e22 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:03.346087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752eb84-60c4-4efc-bf5e-91259b238e22 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:03.440762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49009f-d0e9-4b8d-82d2-84bc9c87538a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:03.441571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49009f-d0e9-4b8d-82d2-84bc9c87538a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:03.441860 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49009f-d0e9-4b8d-82d2-84bc9c87538a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:03.442134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49009f-d0e9-4b8d-82d2-84bc9c87538a admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:03.443049 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49009f-d0e9-4b8d-82d2-84bc9c87538a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:03.445117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49009f-d0e9-4b8d-82d2-84bc9c87538a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:03.555210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1207/2434] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:53:03 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:03.556062 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a1bba-7207-409a-899e-56fbe08a4bfa admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:03.568072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a1bba-7207-409a-899e-56fbe08a4bfa admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:03.574940 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafd990-47b1-473a-b876-bf63946d53ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:03.575636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafd990-47b1-473a-b876-bf63946d53ab admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:03.575915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafd990-47b1-473a-b876-bf63946d53ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:03.576706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fafd990-47b1-473a-b876-bf63946d53ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:03.578888 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fafd990-47b1-473a-b876-bf63946d53ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:03.581309 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fafd990-47b1-473a-b876-bf63946d53ab admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:03.596644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1228/2435] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:53:03.600464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49009f-d0e9-4b8d-82d2-84bc9c87538a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:03.611616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49009f-d0e9-4b8d-82d2-84bc9c87538a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:03.617341 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558aa9ae-b4d1-4894-8974-ce648c8c0aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:03.618054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aa9ae-b4d1-4894-8974-ce648c8c0aaf admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:03.618392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aa9ae-b4d1-4894-8974-ce648c8c0aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:03.618669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aa9ae-b4d1-4894-8974-ce648c8c0aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:03.619409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558aa9ae-b4d1-4894-8974-ce648c8c0aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:03.621055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558aa9ae-b4d1-4894-8974-ce648c8c0aaf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:03.632806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1208/2436] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:03.637274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafd990-47b1-473a-b876-bf63946d53ab admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:03.651547 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fafd990-47b1-473a-b876-bf63946d53ab admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:03.654218 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb9d29-8bba-4a25-b1e3-91f6a279ce33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:03.655213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb9d29-8bba-4a25-b1e3-91f6a279ce33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:03.655577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb9d29-8bba-4a25-b1e3-91f6a279ce33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:03.655934 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb9d29-8bba-4a25-b1e3-91f6a279ce33 admin admin] PATH_INFO: `/v3/projects/4fe102d227ea4f60bdf22c20cd028341/users/81d0641fc86a43f7a6c772add1e2ae48/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:03.661210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87de8b-085a-42a7-b434-6dbd12303ff9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:03.661499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87de8b-085a-42a7-b434-6dbd12303ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:03.661773 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87de8b-085a-42a7-b434-6dbd12303ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:03.676523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb9d29-8bba-4a25-b1e3-91f6a279ce33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe102d227ea4f60bdf22c20cd028341, user_id=81d0641fc86a43f7a6c772add1e2ae48, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:03.679456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb9d29-8bba-4a25-b1e3-91f6a279ce33 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:03.682062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1a87de8b-085a-42a7-b434-6dbd12303ff9 None None] MFA Rules not processed for user `f2d1f86034b94e289bb1fc754f02f4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:03.683785 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1a87de8b-085a-42a7-b434-6dbd12303ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:03.744697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1229/2437] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:03 2022] PUT /identity/v3/projects/4fe102d227ea4f60bdf22c20cd028341/users/81d0641fc86a43f7a6c772add1e2ae48/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:03.750172 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b12fa9-b8b0-41ee-8402-b3401d7f4ad6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:03.750503 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b12fa9-b8b0-41ee-8402-b3401d7f4ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:03.750824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b12fa9-b8b0-41ee-8402-b3401d7f4ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:03.753034 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1209/2438] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:03.771237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-50b12fa9-b8b0-41ee-8402-b3401d7f4ad6 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:03.772483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-50b12fa9-b8b0-41ee-8402-b3401d7f4ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:03.852778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1230/2439] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:03.865642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87de8b-085a-42a7-b434-6dbd12303ff9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:03.867144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1a87de8b-085a-42a7-b434-6dbd12303ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:03.943093 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87de8b-085a-42a7-b434-6dbd12303ff9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:03.995293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2d3f6-9dba-44f6-b9d6-a98f52503798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:03.995864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2d3f6-9dba-44f6-b9d6-a98f52503798 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:03.996148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2d3f6-9dba-44f6-b9d6-a98f52503798 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:03.996424 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2d3f6-9dba-44f6-b9d6-a98f52503798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:04.006404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2d3f6-9dba-44f6-b9d6-a98f52503798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:04.008024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2d3f6-9dba-44f6-b9d6-a98f52503798 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:04.019531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1210/2440] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:04.251152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:05.573376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee2e76-92a9-485d-af48-fdee63d98ed9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:05.577464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee2e76-92a9-485d-af48-fdee63d98ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:05.577464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee2e76-92a9-485d-af48-fdee63d98ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:05.580692 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:05.615257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-63ee2e76-92a9-485d-af48-fdee63d98ed9 None None] MFA Rules not processed for user `81d0641fc86a43f7a6c772add1e2ae48`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:05.615257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-63ee2e76-92a9-485d-af48-fdee63d98ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:05.683754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1231/2441] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:05.695375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2d3f6-9dba-44f6-b9d6-a98f52503798 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:05.701574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5dc2d3f6-9dba-44f6-b9d6-a98f52503798 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:05.765215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2d3f6-9dba-44f6-b9d6-a98f52503798 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:05.769660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a1313-d71c-46dd-8380-035c073f52c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:05.770308 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a1313-d71c-46dd-8380-035c073f52c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:05.770587 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a1313-d71c-46dd-8380-035c073f52c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:05.770862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a1313-d71c-46dd-8380-035c073f52c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:05.779657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a1313-d71c-46dd-8380-035c073f52c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:05.781210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a1313-d71c-46dd-8380-035c073f52c6 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:05.790330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1211/2442] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:07.314076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee2e76-92a9-485d-af48-fdee63d98ed9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:07.315558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-63ee2e76-92a9-485d-af48-fdee63d98ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:07.391606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee2e76-92a9-485d-af48-fdee63d98ed9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:07.397084 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee22708-b4d3-44ef-8b8d-f6b152d77c6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:07.397721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee22708-b4d3-44ef-8b8d-f6b152d77c6e service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:07.397998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee22708-b4d3-44ef-8b8d-f6b152d77c6e service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:07.398269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee22708-b4d3-44ef-8b8d-f6b152d77c6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:07.400251 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1ee22708-b4d3-44ef-8b8d-f6b152d77c6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:07.442577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee22708-b4d3-44ef-8b8d-f6b152d77c6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:07.442577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee22708-b4d3-44ef-8b8d-f6b152d77c6e service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:07.442577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1232/2443] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:08.528046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233f6d4-f912-4ce3-8952-01ee57c8efaa None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:08.528696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233f6d4-f912-4ce3-8952-01ee57c8efaa None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:08.528696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233f6d4-f912-4ce3-8952-01ee57c8efaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:08.546542 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-3233f6d4-f912-4ce3-8952-01ee57c8efaa None None] MFA Rules not processed for user `81d0641fc86a43f7a6c772add1e2ae48`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:08.547851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3233f6d4-f912-4ce3-8952-01ee57c8efaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:08.555317 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1212/2444] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:53:13.411018 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee22708-b4d3-44ef-8b8d-f6b152d77c6e service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:13.412472 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1ee22708-b4d3-44ef-8b8d-f6b152d77c6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:13.428959 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee22708-b4d3-44ef-8b8d-f6b152d77c6e service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:13.433836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:13.434912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:13.435339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:13.435814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] PATH_INFO: `/v3/users/7a2578d5eb784a56819aa1b176a12b08` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:13.437872 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a2578d5eb784a56819aa1b176a12b08)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:13.439980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:13.769635 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a2578d5eb784a56819aa1b176a12b08'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:13.782820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a2578d5eb784a56819aa1b176a12b08'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:13.789931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a2578d5eb784a56819aa1b176a12b08'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:13.792651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:13.873577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a2578d5eb784a56819aa1b176a12b08'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:13.874475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a2578d5eb784a56819aa1b176a12b08'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:13.881344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1233/2445] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:53:13 2022] DELETE /identity/v3/users/7a2578d5eb784a56819aa1b176a12b08 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:13.887134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4d6a-2878-4c46-a9d1-7e3d77852cfc None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:13.887888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4d6a-2878-4c46-a9d1-7e3d77852cfc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:13.888075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4d6a-2878-4c46-a9d1-7e3d77852cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:13.910961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d70f4d6a-2878-4c46-a9d1-7e3d77852cfc None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:13.912186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d70f4d6a-2878-4c46-a9d1-7e3d77852cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:13.986393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1213/2446] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:53:13.996623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:13.998194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:14.058414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eb8fc-a5f3-4add-8063-88b20bba92a2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:14.062068 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df3b87-6911-467c-863b-738ad0a0dea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:14.062720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3b87-6911-467c-863b-738ad0a0dea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:14.063005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3b87-6911-467c-863b-738ad0a0dea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:14.063278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df3b87-6911-467c-863b-738ad0a0dea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:14.072554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df3b87-6911-467c-863b-738ad0a0dea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:14.073994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df3b87-6911-467c-863b-738ad0a0dea4 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:14.084220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1234/2447] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:14.698920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f4d6a-2878-4c46-a9d1-7e3d77852cfc None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:14.700367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d70f4d6a-2878-4c46-a9d1-7e3d77852cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:14.713635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f4d6a-2878-4c46-a9d1-7e3d77852cfc None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:14.720923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:14.722074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:14.722498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:14.722927 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] PATH_INFO: `/v3/projects/a29e46896ebd4b739acb25872ed5ab2c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:14.724520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a29e46896ebd4b739acb25872ed5ab2c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:14.728293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:15.134074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a29e46896ebd4b739acb25872ed5ab2c', 'request_id': 'req-9f4a1cfa-6007-4a38-80c3-b12d224ac453'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:15.141708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a29e46896ebd4b739acb25872ed5ab2c', 'request_id': 'req-9f4a1cfa-6007-4a38-80c3-b12d224ac453'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:15.147807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a29e46896ebd4b739acb25872ed5ab2c', 'request_id': 'req-9f4a1cfa-6007-4a38-80c3-b12d224ac453'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:15.151982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:15.240053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] The token cache is being invalidate because project a29e46896ebd4b739acb25872ed5ab2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:53:15.240441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:15.241496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9f4a1cfa-6007-4a38-80c3-b12d224ac453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:15.243099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1214/2448] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:53:14 2022] DELETE /identity/v3/projects/a29e46896ebd4b739acb25872ed5ab2c => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:15.247993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5bd82-e714-49c9-8fea-f0f341827c50 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:15.248492 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5bd82-e714-49c9-8fea-f0f341827c50 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:15.248602 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5bd82-e714-49c9-8fea-f0f341827c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:15.267612 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-80c5bd82-e714-49c9-8fea-f0f341827c50 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:15.268808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-80c5bd82-e714-49c9-8fea-f0f341827c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:15.345232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1235/2449] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:15.354786 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d59f3c-dc44-498d-b902-7f62111ecec6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:15.355100 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d59f3c-dc44-498d-b902-7f62111ecec6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:15.355378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d59f3c-dc44-498d-b902-7f62111ecec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:15.376429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-28d59f3c-dc44-498d-b902-7f62111ecec6 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:15.379619 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-28d59f3c-dc44-498d-b902-7f62111ecec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:15.386221 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1215/2450] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:15.390673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5bd82-e714-49c9-8fea-f0f341827c50 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:15.402455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5bd82-e714-49c9-8fea-f0f341827c50 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:15.408380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43eacc-1cb5-49ff-a23d-60409c98f2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:15.409093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43eacc-1cb5-49ff-a23d-60409c98f2bb admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:15.409381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43eacc-1cb5-49ff-a23d-60409c98f2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:15.409661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43eacc-1cb5-49ff-a23d-60409c98f2bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:15.410557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43eacc-1cb5-49ff-a23d-60409c98f2bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:53:15.411072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43eacc-1cb5-49ff-a23d-60409c98f2bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:15.412569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43eacc-1cb5-49ff-a23d-60409c98f2bb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:15.426967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1236/2451] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:53:15.432535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d59f3c-dc44-498d-b902-7f62111ecec6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:15.442581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d59f3c-dc44-498d-b902-7f62111ecec6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:15.450272 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed997be1-cb2b-43e9-bafb-259c224e2ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:15.450938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed997be1-cb2b-43e9-bafb-259c224e2ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:15.451222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed997be1-cb2b-43e9-bafb-259c224e2ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:15.451497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed997be1-cb2b-43e9-bafb-259c224e2ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:15.452215 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed997be1-cb2b-43e9-bafb-259c224e2ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:15.454295 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed997be1-cb2b-43e9-bafb-259c224e2ba3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:15.534717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1216/2452] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:53:15 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:15.538826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43eacc-1cb5-49ff-a23d-60409c98f2bb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:15.549821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43eacc-1cb5-49ff-a23d-60409c98f2bb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:15.626534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a4fa4-90a9-4c68-9682-cf27debf5baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:15.627209 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a4fa4-90a9-4c68-9682-cf27debf5baf admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:15.627494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a4fa4-90a9-4c68-9682-cf27debf5baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:15.627775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a4fa4-90a9-4c68-9682-cf27debf5baf admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:15.628467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a4fa4-90a9-4c68-9682-cf27debf5baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:15.630456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a4fa4-90a9-4c68-9682-cf27debf5baf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:15.731786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1237/2453] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:53:15 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:15.736019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed997be1-cb2b-43e9-bafb-259c224e2ba3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:15.747047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed997be1-cb2b-43e9-bafb-259c224e2ba3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:15.751856 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2c689-659b-4089-9a04-075e7fb054c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:15.752525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2c689-659b-4089-9a04-075e7fb054c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:15.752810 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2c689-659b-4089-9a04-075e7fb054c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:15.753128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2c689-659b-4089-9a04-075e7fb054c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:15.753904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2c689-659b-4089-9a04-075e7fb054c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:15.755514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2c689-659b-4089-9a04-075e7fb054c1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:15.766854 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1217/2454] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:15.770799 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a4fa4-90a9-4c68-9682-cf27debf5baf admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:15.781608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a4fa4-90a9-4c68-9682-cf27debf5baf admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:15.786279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710694d-12d7-48bc-ac73-3293d685dc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:15.786937 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710694d-12d7-48bc-ac73-3293d685dc41 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:15.787283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710694d-12d7-48bc-ac73-3293d685dc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:15.787591 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710694d-12d7-48bc-ac73-3293d685dc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:15.788326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710694d-12d7-48bc-ac73-3293d685dc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:15.789963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710694d-12d7-48bc-ac73-3293d685dc41 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:15.800487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1238/2455] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:15.804657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2c689-659b-4089-9a04-075e7fb054c1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:15.815222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2c689-659b-4089-9a04-075e7fb054c1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:15.819885 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f239b48-f13f-48f8-828a-1c3ea1df07fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:15.820762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f239b48-f13f-48f8-828a-1c3ea1df07fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:15.821074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f239b48-f13f-48f8-828a-1c3ea1df07fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:15.821353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f239b48-f13f-48f8-828a-1c3ea1df07fc admin admin] PATH_INFO: `/v3/projects/a0aa59ff7f5a460daa866b1f3e6dae1a/users/ab4d3667826148d390eda90bb8baf081/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:15.836104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f239b48-f13f-48f8-828a-1c3ea1df07fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0aa59ff7f5a460daa866b1f3e6dae1a, user_id=ab4d3667826148d390eda90bb8baf081, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:15.838807 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f239b48-f13f-48f8-828a-1c3ea1df07fc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:15.918827 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1218/2456] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:15 2022] PUT /identity/v3/projects/a0aa59ff7f5a460daa866b1f3e6dae1a/users/ab4d3667826148d390eda90bb8baf081/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:15.924421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6ab17-e20c-4e74-9418-f32471ebf820 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:15.930793 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6ab17-e20c-4e74-9418-f32471ebf820 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:15.930793 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6ab17-e20c-4e74-9418-f32471ebf820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:15.931895 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:15.944131 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-18e6ab17-e20c-4e74-9418-f32471ebf820 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:15.945408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-18e6ab17-e20c-4e74-9418-f32471ebf820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:16.005789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1239/2457] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:16.016749 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f239b48-f13f-48f8-828a-1c3ea1df07fc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:16.016983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0f239b48-f13f-48f8-828a-1c3ea1df07fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:16.077669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f239b48-f13f-48f8-828a-1c3ea1df07fc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:16.081613 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22d9aa-5949-4991-a331-a0e4178d14d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:16.082696 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d9aa-5949-4991-a331-a0e4178d14d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:16.082991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d9aa-5949-4991-a331-a0e4178d14d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:16.083278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22d9aa-5949-4991-a331-a0e4178d14d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:16.093814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22d9aa-5949-4991-a331-a0e4178d14d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:16.095960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22d9aa-5949-4991-a331-a0e4178d14d8 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:16.114852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1219/2458] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:22.290883 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9960ab4-ec4b-4be9-aa1c-4064c6305016 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:22.291427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9960ab4-ec4b-4be9-aa1c-4064c6305016 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:22.291427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9960ab4-ec4b-4be9-aa1c-4064c6305016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:22.363569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a9960ab4-ec4b-4be9-aa1c-4064c6305016 None None] MFA Rules not processed for user `ab4d3667826148d390eda90bb8baf081`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:22.365061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a9960ab4-ec4b-4be9-aa1c-4064c6305016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:22.475033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1240/2459] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:22.479537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22d9aa-5949-4991-a331-a0e4178d14d8 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:22.491600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22d9aa-5949-4991-a331-a0e4178d14d8 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:22.503404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a032080-ee4c-4afd-aacd-2d82c1943956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:22.504098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a032080-ee4c-4afd-aacd-2d82c1943956 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:22.504379 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a032080-ee4c-4afd-aacd-2d82c1943956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:22.504652 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a032080-ee4c-4afd-aacd-2d82c1943956 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:22.506485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a032080-ee4c-4afd-aacd-2d82c1943956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:22.508709 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a032080-ee4c-4afd-aacd-2d82c1943956 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:22.552973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9960ab4-ec4b-4be9-aa1c-4064c6305016 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:22.554477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a9960ab4-ec4b-4be9-aa1c-4064c6305016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:22.577778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9960ab4-ec4b-4be9-aa1c-4064c6305016 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:22.577778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda09cf2-aa52-45ac-8303-ceee8a17087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:22.577778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda09cf2-aa52-45ac-8303-ceee8a17087c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:22.577778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda09cf2-aa52-45ac-8303-ceee8a17087c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:22.578263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda09cf2-aa52-45ac-8303-ceee8a17087c admin admin] PATH_INFO: `/v3/users/988755104aea4d3cb784ff0c397e75fd` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:22.578263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda09cf2-aa52-45ac-8303-ceee8a17087c admin admin] RBAC: Authorizing `identity:delete_user(user_id=988755104aea4d3cb784ff0c397e75fd)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:22.579199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda09cf2-aa52-45ac-8303-ceee8a17087c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:22.606491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1220/2460] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:53:22 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:22.610713 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a032080-ee4c-4afd-aacd-2d82c1943956 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:22.686174 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a032080-ee4c-4afd-aacd-2d82c1943956 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:22.691557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cec48-21e2-4ff8-9907-da0cccded690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:22.692431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cec48-21e2-4ff8-9907-da0cccded690 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:22.692824 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cec48-21e2-4ff8-9907-da0cccded690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:22.693267 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18cec48-21e2-4ff8-9907-da0cccded690 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:22.694121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18cec48-21e2-4ff8-9907-da0cccded690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:22.696237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18cec48-21e2-4ff8-9907-da0cccded690 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:22.769821 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bda09cf2-aa52-45ac-8303-ceee8a17087c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988755104aea4d3cb784ff0c397e75fd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:22.790436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1221/2461] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:53:22 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:53:22.795176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cec48-21e2-4ff8-9907-da0cccded690 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:22.801179 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:22.815488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18cec48-21e2-4ff8-9907-da0cccded690 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:22.827018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90c45e-9d82-4514-ac17-85d7c528b957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:22.829397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c45e-9d82-4514-ac17-85d7c528b957 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:22.832775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c45e-9d82-4514-ac17-85d7c528b957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:22.832775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c45e-9d82-4514-ac17-85d7c528b957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:22.832775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90c45e-9d82-4514-ac17-85d7c528b957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:22.835292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90c45e-9d82-4514-ac17-85d7c528b957 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:22.849872 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bda09cf2-aa52-45ac-8303-ceee8a17087c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988755104aea4d3cb784ff0c397e75fd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:22.861222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1222/2462] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:53:22.863170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bda09cf2-aa52-45ac-8303-ceee8a17087c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988755104aea4d3cb784ff0c397e75fd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:22.864442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90c45e-9d82-4514-ac17-85d7c528b957 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:22.889686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90c45e-9d82-4514-ac17-85d7c528b957 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:22.901651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbce892-e0e7-47ac-a154-e6e84b6e06ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:22.903005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbce892-e0e7-47ac-a154-e6e84b6e06ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:22.903246 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbce892-e0e7-47ac-a154-e6e84b6e06ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:22.903585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbce892-e0e7-47ac-a154-e6e84b6e06ff admin admin] PATH_INFO: `/v3/projects/b3c553fb947148fcbfcdab882d77ae3d/users/dc1e3642b7a24808ba8e4160a0f66ba7/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:22.935681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbce892-e0e7-47ac-a154-e6e84b6e06ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c553fb947148fcbfcdab882d77ae3d, user_id=dc1e3642b7a24808ba8e4160a0f66ba7, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:22.942607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbce892-e0e7-47ac-a154-e6e84b6e06ff admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:22.942909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bda09cf2-aa52-45ac-8303-ceee8a17087c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988755104aea4d3cb784ff0c397e75fd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:22.943558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bda09cf2-aa52-45ac-8303-ceee8a17087c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988755104aea4d3cb784ff0c397e75fd'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:22.957805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1241/2463] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:53:22 2022] DELETE /identity/v3/users/988755104aea4d3cb784ff0c397e75fd => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:22.963083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95045e3-e0f7-4f01-a294-5b646d270639 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:22.963383 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95045e3-e0f7-4f01-a294-5b646d270639 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:22.963716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95045e3-e0f7-4f01-a294-5b646d270639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:22.974246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:22.997112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b95045e3-e0f7-4f01-a294-5b646d270639 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:22.998566 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b95045e3-e0f7-4f01-a294-5b646d270639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:23.016024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1223/2464] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:22 2022] PUT /identity/v3/projects/b3c553fb947148fcbfcdab882d77ae3d/users/dc1e3642b7a24808ba8e4160a0f66ba7/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:23.019902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbce892-e0e7-47ac-a154-e6e84b6e06ff admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:23.022192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6cbce892-e0e7-47ac-a154-e6e84b6e06ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:23.099278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1242/2465] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:53:23.101106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95045e3-e0f7-4f01-a294-5b646d270639 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:23.102553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b95045e3-e0f7-4f01-a294-5b646d270639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:23.129535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbce892-e0e7-47ac-a154-e6e84b6e06ff admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:23.137853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0aec0-1c1d-49b8-87f5-1a9528c3aaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:23.139327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0aec0-1c1d-49b8-87f5-1a9528c3aaed admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:23.140520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0aec0-1c1d-49b8-87f5-1a9528c3aaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:23.140813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0aec0-1c1d-49b8-87f5-1a9528c3aaed admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:23.142067 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0aec0-1c1d-49b8-87f5-1a9528c3aaed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:23.144201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0aec0-1c1d-49b8-87f5-1a9528c3aaed admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:23.158550 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1224/2466] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:23.162451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0aec0-1c1d-49b8-87f5-1a9528c3aaed admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:23.174703 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0aec0-1c1d-49b8-87f5-1a9528c3aaed admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:23.180786 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e02bb-80ce-4b86-95c2-06c39bc0abf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:23.181729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e02bb-80ce-4b86-95c2-06c39bc0abf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:23.182019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e02bb-80ce-4b86-95c2-06c39bc0abf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:23.182298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e02bb-80ce-4b86-95c2-06c39bc0abf9 admin admin] PATH_INFO: `/v3/projects/b3c553fb947148fcbfcdab882d77ae3d/users/dc1e3642b7a24808ba8e4160a0f66ba7/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:23.183346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95045e3-e0f7-4f01-a294-5b646d270639 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:23.184390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e02bb-80ce-4b86-95c2-06c39bc0abf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c553fb947148fcbfcdab882d77ae3d, user_id=dc1e3642b7a24808ba8e4160a0f66ba7, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:23.187853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e02bb-80ce-4b86-95c2-06c39bc0abf9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:23.201449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1225/2467] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:23 2022] PUT /identity/v3/projects/b3c553fb947148fcbfcdab882d77ae3d/users/dc1e3642b7a24808ba8e4160a0f66ba7/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:53:23.247315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90f3c9-4c7f-4d02-8b89-0f2d9c6d2d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:23.248503 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90f3c9-4c7f-4d02-8b89-0f2d9c6d2d52 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:23.248503 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90f3c9-4c7f-4d02-8b89-0f2d9c6d2d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:23.248745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90f3c9-4c7f-4d02-8b89-0f2d9c6d2d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:23.262435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90f3c9-4c7f-4d02-8b89-0f2d9c6d2d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:23.264052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90f3c9-4c7f-4d02-8b89-0f2d9c6d2d52 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:23.283305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1243/2468] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:53:24.220112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e02bb-80ce-4b86-95c2-06c39bc0abf9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:24.222524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7c5e02bb-80ce-4b86-95c2-06c39bc0abf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:24.324796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e02bb-80ce-4b86-95c2-06c39bc0abf9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:24.330210 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:24.331276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:24.331642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:24.332253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] PATH_INFO: `/v3/projects/90e84f440a9241bb87fc5732a9553a57` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:24.333782 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] RBAC: Authorizing `identity:delete_project(project_id=90e84f440a9241bb87fc5732a9553a57)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:24.336246 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:24.467351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90e84f440a9241bb87fc5732a9553a57', 'request_id': 'req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:24.471522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:24.478429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90e84f440a9241bb87fc5732a9553a57', 'request_id': 'req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:24.486360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90e84f440a9241bb87fc5732a9553a57', 'request_id': 'req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:24.627242 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] The token cache is being invalidate because project 90e84f440a9241bb87fc5732a9553a57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:53:24.627242 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:24.627242 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:24.632004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1226/2469] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:53:24 2022] DELETE /identity/v3/projects/90e84f440a9241bb87fc5732a9553a57 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:24.633432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90f3c9-4c7f-4d02-8b89-0f2d9c6d2d52 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:24.644937 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90f3c9-4c7f-4d02-8b89-0f2d9c6d2d52 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:24.754820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6fa8f3-389c-4972-93c7-1cf9a939ea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:24.756174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fa8f3-389c-4972-93c7-1cf9a939ea42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:24.756521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fa8f3-389c-4972-93c7-1cf9a939ea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:24.756904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6fa8f3-389c-4972-93c7-1cf9a939ea42 admin admin] PATH_INFO: `/v3/users/d95169a9c138445baa7686246e0a9abc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:24.761057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fa8f3-389c-4972-93c7-1cf9a939ea42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d95169a9c138445baa7686246e0a9abc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:24.765407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6fa8f3-389c-4972-93c7-1cf9a939ea42 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:24.774454 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:24.984289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-da6fa8f3-389c-4972-93c7-1cf9a939ea42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd95169a9c138445baa7686246e0a9abc'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:24.999986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-da6fa8f3-389c-4972-93c7-1cf9a939ea42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd95169a9c138445baa7686246e0a9abc'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:25.008867 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-da6fa8f3-389c-4972-93c7-1cf9a939ea42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd95169a9c138445baa7686246e0a9abc'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:25.068085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-da6fa8f3-389c-4972-93c7-1cf9a939ea42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd95169a9c138445baa7686246e0a9abc'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:25.068718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-da6fa8f3-389c-4972-93c7-1cf9a939ea42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd95169a9c138445baa7686246e0a9abc'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:25.075558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1244/2470] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:53:24 2022] DELETE /identity/v3/users/d95169a9c138445baa7686246e0a9abc => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:25.586940 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:25.588562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:25.618319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6710-ba46-4fdf-8e0b-59aad279a97e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:25.695540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:25.696523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:25.697047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:25.697327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] PATH_INFO: `/v3/projects/86638b478b024df29c0cb8e4cbf583c7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:25.700598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86638b478b024df29c0cb8e4cbf583c7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:25.705712 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:25.912670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86638b478b024df29c0cb8e4cbf583c7', 'request_id': 'req-0381fed7-20dc-436e-8b7f-bf117a922698'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:25.928447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86638b478b024df29c0cb8e4cbf583c7', 'request_id': 'req-0381fed7-20dc-436e-8b7f-bf117a922698'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:25.948759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86638b478b024df29c0cb8e4cbf583c7', 'request_id': 'req-0381fed7-20dc-436e-8b7f-bf117a922698'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:26.038496 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] The token cache is being invalidate because project 86638b478b024df29c0cb8e4cbf583c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:53:26.038799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:26.039486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0381fed7-20dc-436e-8b7f-bf117a922698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:26.055516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1227/2471] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:53:25 2022] DELETE /identity/v3/projects/86638b478b024df29c0cb8e4cbf583c7 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:26.060341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0da859-324d-48a1-bdc7-880212648fa9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:26.061943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0da859-324d-48a1-bdc7-880212648fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:26.063059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0da859-324d-48a1-bdc7-880212648fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:26.110792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2e0da859-324d-48a1-bdc7-880212648fa9 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:26.110792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2e0da859-324d-48a1-bdc7-880212648fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:26.226967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1245/2472] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:53:26.233022 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4e2e-11d9-43fe-8489-d924ac8fd8fc None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:26.233946 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4e2e-11d9-43fe-8489-d924ac8fd8fc None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:26.234236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f4e2e-11d9-43fe-8489-d924ac8fd8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:26.269990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1c1f4e2e-11d9-43fe-8489-d924ac8fd8fc None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:26.273573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1c1f4e2e-11d9-43fe-8489-d924ac8fd8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:26.296504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1228/2473] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:26.302317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0da859-324d-48a1-bdc7-880212648fa9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:26.328241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0da859-324d-48a1-bdc7-880212648fa9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:26.343657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be6351-df06-41d6-8779-2b8f988313ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:26.344573 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be6351-df06-41d6-8779-2b8f988313ca admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:26.345284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be6351-df06-41d6-8779-2b8f988313ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:26.345750 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be6351-df06-41d6-8779-2b8f988313ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:26.348190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be6351-df06-41d6-8779-2b8f988313ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:53:26.349799 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be6351-df06-41d6-8779-2b8f988313ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:26.354788 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be6351-df06-41d6-8779-2b8f988313ca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:26.381807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1246/2474] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:53:26.386348 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f4e2e-11d9-43fe-8489-d924ac8fd8fc None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:26.403682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f4e2e-11d9-43fe-8489-d924ac8fd8fc None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:26.428557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88471187-14ea-4b1f-93e3-580e2c84cec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:26.430325 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88471187-14ea-4b1f-93e3-580e2c84cec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:26.430832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88471187-14ea-4b1f-93e3-580e2c84cec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:26.432046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88471187-14ea-4b1f-93e3-580e2c84cec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:26.434005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88471187-14ea-4b1f-93e3-580e2c84cec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:26.437719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88471187-14ea-4b1f-93e3-580e2c84cec7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:26.580384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1229/2475] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:53:26 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:26.585609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be6351-df06-41d6-8779-2b8f988313ca admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:26.615112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be6351-df06-41d6-8779-2b8f988313ca admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:26.720918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a04e7-0f23-4b52-a670-5997ac2a350f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:26.723428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a04e7-0f23-4b52-a670-5997ac2a350f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:26.723428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a04e7-0f23-4b52-a670-5997ac2a350f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:26.723428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a04e7-0f23-4b52-a670-5997ac2a350f admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:26.723720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a04e7-0f23-4b52-a670-5997ac2a350f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:26.725906 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a04e7-0f23-4b52-a670-5997ac2a350f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:26.836206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:26.840435 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1247/2476] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:53:26 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:26.844631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88471187-14ea-4b1f-93e3-580e2c84cec7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:26.860837 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88471187-14ea-4b1f-93e3-580e2c84cec7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:26.870337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5d25c-2c4b-403a-a389-7692ebc02639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:26.871279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5d25c-2c4b-403a-a389-7692ebc02639 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:26.871655 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5d25c-2c4b-403a-a389-7692ebc02639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:26.872019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5d25c-2c4b-403a-a389-7692ebc02639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:26.872828 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5d25c-2c4b-403a-a389-7692ebc02639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:26.874729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5d25c-2c4b-403a-a389-7692ebc02639 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:26.887439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1230/2477] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:53:26.893054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a04e7-0f23-4b52-a670-5997ac2a350f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:26.903982 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a04e7-0f23-4b52-a670-5997ac2a350f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:26.917396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd3fe1-4182-44d6-8184-4f31ecd3f6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:26.917396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3fe1-4182-44d6-8184-4f31ecd3f6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:26.917396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3fe1-4182-44d6-8184-4f31ecd3f6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:26.917396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3fe1-4182-44d6-8184-4f31ecd3f6e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:26.917765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd3fe1-4182-44d6-8184-4f31ecd3f6e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:26.918536 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd3fe1-4182-44d6-8184-4f31ecd3f6e3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:26.946373 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1248/2478] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:26.949089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5d25c-2c4b-403a-a389-7692ebc02639 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:26.983161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5d25c-2c4b-403a-a389-7692ebc02639 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:27.004230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac58f8f-a6e4-4a0c-9bb2-0a8973d44f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:27.009358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac58f8f-a6e4-4a0c-9bb2-0a8973d44f39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:27.009358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac58f8f-a6e4-4a0c-9bb2-0a8973d44f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:27.009358 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac58f8f-a6e4-4a0c-9bb2-0a8973d44f39 admin admin] PATH_INFO: `/v3/projects/9e62eb11e3e34168a6673e0cdd478a30/users/503080c18a084dc2980311c4b80a0ce7/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:27.038157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac58f8f-a6e4-4a0c-9bb2-0a8973d44f39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e62eb11e3e34168a6673e0cdd478a30, user_id=503080c18a084dc2980311c4b80a0ce7, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:27.042522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac58f8f-a6e4-4a0c-9bb2-0a8973d44f39 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:27.159753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1231/2479] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:26 2022] PUT /identity/v3/projects/9e62eb11e3e34168a6673e0cdd478a30/users/503080c18a084dc2980311c4b80a0ce7/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:27.172347 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7305880-b248-4abd-812b-9c7b546909ce None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:27.173628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7305880-b248-4abd-812b-9c7b546909ce None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:27.174125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7305880-b248-4abd-812b-9c7b546909ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:27.207625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a7305880-b248-4abd-812b-9c7b546909ce None None] MFA Rules not processed for user `503080c18a084dc2980311c4b80a0ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:27.209244 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a7305880-b248-4abd-812b-9c7b546909ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:27.287791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1249/2480] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:27.292299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac58f8f-a6e4-4a0c-9bb2-0a8973d44f39 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:27.302645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:27.365938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac58f8f-a6e4-4a0c-9bb2-0a8973d44f39 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:27.370997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4ff7f-e690-45f1-9dea-44402326bea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:27.371822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4ff7f-e690-45f1-9dea-44402326bea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:27.372117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4ff7f-e690-45f1-9dea-44402326bea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:27.372391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4ff7f-e690-45f1-9dea-44402326bea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:27.373123 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4ff7f-e690-45f1-9dea-44402326bea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:27.375674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4ff7f-e690-45f1-9dea-44402326bea1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:27.576917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7305880-b248-4abd-812b-9c7b546909ce None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:27.578124 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1232/2481] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:53:27 2022] POST /identity/v3/projects => generated 385 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:27.598529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7305880-b248-4abd-812b-9c7b546909ce None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:27.695629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8b947-afc2-4dcf-a972-8182ffec9a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:27.697257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8b947-afc2-4dcf-a972-8182ffec9a6f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:27.697776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8b947-afc2-4dcf-a972-8182ffec9a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:27.698188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8b947-afc2-4dcf-a972-8182ffec9a6f admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:27.699511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8b947-afc2-4dcf-a972-8182ffec9a6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:27.703135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8b947-afc2-4dcf-a972-8182ffec9a6f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:27.774938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1250/2482] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:53:27 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:27.776757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4ff7f-e690-45f1-9dea-44402326bea1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:27.790752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4ff7f-e690-45f1-9dea-44402326bea1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:27.799097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de800dc4-a3f6-4fa2-abe1-47ea1b16e5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:27.799805 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800dc4-a3f6-4fa2-abe1-47ea1b16e5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:27.800096 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800dc4-a3f6-4fa2-abe1-47ea1b16e5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:27.800405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800dc4-a3f6-4fa2-abe1-47ea1b16e5c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:27.801378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de800dc4-a3f6-4fa2-abe1-47ea1b16e5c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:27.809192 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de800dc4-a3f6-4fa2-abe1-47ea1b16e5c7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:27.826633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1233/2483] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:27.827909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8b947-afc2-4dcf-a972-8182ffec9a6f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:27.843140 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8b947-afc2-4dcf-a972-8182ffec9a6f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:27.854618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1f872-d3ac-4028-81f1-2c6e78cbe41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:27.855808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1f872-d3ac-4028-81f1-2c6e78cbe41d admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:27.856394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1f872-d3ac-4028-81f1-2c6e78cbe41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:27.856854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de1f872-d3ac-4028-81f1-2c6e78cbe41d admin admin] PATH_INFO: `/v3/projects/9eb17174a4d1402488588f86f5bec0bc/users/56e5dff40f13467fba67f990802c6558/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:27.891233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1f872-d3ac-4028-81f1-2c6e78cbe41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb17174a4d1402488588f86f5bec0bc, user_id=56e5dff40f13467fba67f990802c6558, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:27.897613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de1f872-d3ac-4028-81f1-2c6e78cbe41d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:27.992349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1251/2484] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:27 2022] PUT /identity/v3/projects/9eb17174a4d1402488588f86f5bec0bc/users/56e5dff40f13467fba67f990802c6558/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:27.996691 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de800dc4-a3f6-4fa2-abe1-47ea1b16e5c7 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:28.008096 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de800dc4-a3f6-4fa2-abe1-47ea1b16e5c7 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:28.082241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f44d9-ad8e-43fa-9ed3-dcd226dd541d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:28.083099 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f44d9-ad8e-43fa-9ed3-dcd226dd541d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:28.083388 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f44d9-ad8e-43fa-9ed3-dcd226dd541d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:28.084828 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f44d9-ad8e-43fa-9ed3-dcd226dd541d admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:28.085986 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f44d9-ad8e-43fa-9ed3-dcd226dd541d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:28.087661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f44d9-ad8e-43fa-9ed3-dcd226dd541d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:28.099905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1234/2485] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:28.104105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1f872-d3ac-4028-81f1-2c6e78cbe41d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:28.118292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de1f872-d3ac-4028-81f1-2c6e78cbe41d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:28.125802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79eadf-befe-4f5b-a197-e2d721a0e071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:28.126835 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79eadf-befe-4f5b-a197-e2d721a0e071 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:28.127420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79eadf-befe-4f5b-a197-e2d721a0e071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:28.127704 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79eadf-befe-4f5b-a197-e2d721a0e071 admin admin] PATH_INFO: `/v3/projects/9eb17174a4d1402488588f86f5bec0bc/users/56e5dff40f13467fba67f990802c6558/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:28.133087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79eadf-befe-4f5b-a197-e2d721a0e071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb17174a4d1402488588f86f5bec0bc, user_id=56e5dff40f13467fba67f990802c6558, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:28.136034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79eadf-befe-4f5b-a197-e2d721a0e071 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:28.184412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3dda8-c1f5-4ca5-8d96-9a7ae90fdd60 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:28.184651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1252/2486] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:28 2022] PUT /identity/v3/projects/9eb17174a4d1402488588f86f5bec0bc/users/56e5dff40f13467fba67f990802c6558/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:28.185588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3dda8-c1f5-4ca5-8d96-9a7ae90fdd60 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:28.185974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3dda8-c1f5-4ca5-8d96-9a7ae90fdd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:28.225601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-07b3dda8-c1f5-4ca5-8d96-9a7ae90fdd60 None None] MFA Rules not processed for user `56e5dff40f13467fba67f990802c6558`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:28.228084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-07b3dda8-c1f5-4ca5-8d96-9a7ae90fdd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:28.365502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1235/2487] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:28.375578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79eadf-befe-4f5b-a197-e2d721a0e071 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:28.381913 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bd79eadf-befe-4f5b-a197-e2d721a0e071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:28.433525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:28.463129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79eadf-befe-4f5b-a197-e2d721a0e071 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:28.470115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c62d-5621-464f-a85a-fd40fef23346 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:28.471092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c62d-5621-464f-a85a-fd40fef23346 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:28.471740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c62d-5621-464f-a85a-fd40fef23346 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:28.472196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c62d-5621-464f-a85a-fd40fef23346 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:28.484712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1c62d-5621-464f-a85a-fd40fef23346 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:28.486697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1c62d-5621-464f-a85a-fd40fef23346 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:28.513215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1253/2488] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:53:29.874891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3dda8-c1f5-4ca5-8d96-9a7ae90fdd60 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:29.879414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:29.955000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3dda8-c1f5-4ca5-8d96-9a7ae90fdd60 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:29.966803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:29.968401 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:29.968742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:29.969114 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] PATH_INFO: `/v3/users/503080c18a084dc2980311c4b80a0ce7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:29.970380 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=503080c18a084dc2980311c4b80a0ce7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:29.972340 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:30.132642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '503080c18a084dc2980311c4b80a0ce7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:30.141852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '503080c18a084dc2980311c4b80a0ce7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:30.158760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '503080c18a084dc2980311c4b80a0ce7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:30.235868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '503080c18a084dc2980311c4b80a0ce7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:30.236569 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '503080c18a084dc2980311c4b80a0ce7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:30.251700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1236/2489] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:53:29 2022] DELETE /identity/v3/users/503080c18a084dc2980311c4b80a0ce7 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:30.255758 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815f9bb-5f48-415c-85ce-900029785be5 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:30.256301 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815f9bb-5f48-415c-85ce-900029785be5 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:30.257163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815f9bb-5f48-415c-85ce-900029785be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:30.316275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6815f9bb-5f48-415c-85ce-900029785be5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:30.318066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6815f9bb-5f48-415c-85ce-900029785be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:30.429933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1254/2490] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:30.443066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:30.444598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:30.539462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21c44b-2d19-41bb-b0c5-fd2b1e4189d7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:30.539623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9140d-9b7c-46c0-a06c-d1a9bd8eaca8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:30.539800 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21c44b-2d19-41bb-b0c5-fd2b1e4189d7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:30.540109 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21c44b-2d19-41bb-b0c5-fd2b1e4189d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:30.549389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b69ba9-b2e7-4da4-955f-bd77667cdb4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:30.551032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b69ba9-b2e7-4da4-955f-bd77667cdb4f service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:30.551319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b69ba9-b2e7-4da4-955f-bd77667cdb4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:30.551602 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b69ba9-b2e7-4da4-955f-bd77667cdb4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:30.565510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b69ba9-b2e7-4da4-955f-bd77667cdb4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:30.567514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b69ba9-b2e7-4da4-955f-bd77667cdb4f service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:30.577190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6a21c44b-2d19-41bb-b0c5-fd2b1e4189d7 None None] MFA Rules not processed for user `dc1e3642b7a24808ba8e4160a0f66ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:30.578546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6a21c44b-2d19-41bb-b0c5-fd2b1e4189d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:30.601449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1237/2491] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:30.709174 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1255/2492] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:53:30.721790 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b69ba9-b2e7-4da4-955f-bd77667cdb4f service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:30.723735 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-29b69ba9-b2e7-4da4-955f-bd77667cdb4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:30.760090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:30.793769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b69ba9-b2e7-4da4-955f-bd77667cdb4f service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:30.798633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46abaf4b-2cd2-4d21-93b0-0616798c9251 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54c7869983bf4534b3e730af0185eac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:30.799426 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abaf4b-2cd2-4d21-93b0-0616798c9251 service glance] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:30.799598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abaf4b-2cd2-4d21-93b0-0616798c9251 service glance] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:30.800062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46abaf4b-2cd2-4d21-93b0-0616798c9251 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:30.802148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-46abaf4b-2cd2-4d21-93b0-0616798c9251 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:30.887277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abaf4b-2cd2-4d21-93b0-0616798c9251 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:30.888780 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46abaf4b-2cd2-4d21-93b0-0616798c9251 service glance] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:30.900786 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1238/2493] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:31.075651 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21c44b-2d19-41bb-b0c5-fd2b1e4189d7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:31.077806 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6a21c44b-2d19-41bb-b0c5-fd2b1e4189d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:31.093776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21c44b-2d19-41bb-b0c5-fd2b1e4189d7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:31.099469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:31.100417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:31.100706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:31.101175 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] PATH_INFO: `/v3/projects/9e62eb11e3e34168a6673e0cdd478a30` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:31.102432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e62eb11e3e34168a6673e0cdd478a30)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:31.105045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:31.157758 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0261a-679f-4275-977a-5e38ba381f82 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:31.158054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0261a-679f-4275-977a-5e38ba381f82 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:31.158364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0261a-679f-4275-977a-5e38ba381f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:31.176225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e2d0261a-679f-4275-977a-5e38ba381f82 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:31.177543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e2d0261a-679f-4275-977a-5e38ba381f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:31.243568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1239/2494] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:53:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:53:31.254278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e62eb11e3e34168a6673e0cdd478a30', 'request_id': 'req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:31.258689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:31.329313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e62eb11e3e34168a6673e0cdd478a30', 'request_id': 'req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:31.335619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e62eb11e3e34168a6673e0cdd478a30', 'request_id': 'req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:31.347740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] The token cache is being invalidate because project 9e62eb11e3e34168a6673e0cdd478a30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:53:31.348029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:31.348677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:31.350336 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1256/2495] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:53:31 2022] DELETE /identity/v3/projects/9e62eb11e3e34168a6673e0cdd478a30 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:31.355740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0261a-679f-4275-977a-5e38ba381f82 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:31.366451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0261a-679f-4275-977a-5e38ba381f82 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:31.418084 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:31.419844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:31.466066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf81e0-ee02-4587-bb25-7cc3fd9a5f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:31.467154 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf81e0-ee02-4587-bb25-7cc3fd9a5f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:31.468046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf81e0-ee02-4587-bb25-7cc3fd9a5f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:31.468328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf81e0-ee02-4587-bb25-7cc3fd9a5f76 admin admin] PATH_INFO: `/v3/users/56e5dff40f13467fba67f990802c6558` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:31.469931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf81e0-ee02-4587-bb25-7cc3fd9a5f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56e5dff40f13467fba67f990802c6558)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:31.471937 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf81e0-ee02-4587-bb25-7cc3fd9a5f76 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:31.528636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcc1a9-8d8c-4b5a-bae2-0022de9ad042 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:31.534634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd67e17-7dbc-4d60-9201-40efec0b5aad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:31.535323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd67e17-7dbc-4d60-9201-40efec0b5aad service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:31.535608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd67e17-7dbc-4d60-9201-40efec0b5aad service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:31.535889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd67e17-7dbc-4d60-9201-40efec0b5aad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:31.537755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5fd67e17-7dbc-4d60-9201-40efec0b5aad service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:31.560017 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:31.607786 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd67e17-7dbc-4d60-9201-40efec0b5aad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:31.609899 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd67e17-7dbc-4d60-9201-40efec0b5aad service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:31.622950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1257/2496] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:53:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 16:53:31.640048 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9daf81e0-ee02-4587-bb25-7cc3fd9a5f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56e5dff40f13467fba67f990802c6558'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:31.654019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9daf81e0-ee02-4587-bb25-7cc3fd9a5f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56e5dff40f13467fba67f990802c6558'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:31.672759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9daf81e0-ee02-4587-bb25-7cc3fd9a5f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56e5dff40f13467fba67f990802c6558'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:31.809019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9daf81e0-ee02-4587-bb25-7cc3fd9a5f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56e5dff40f13467fba67f990802c6558'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:31.809854 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-9daf81e0-ee02-4587-bb25-7cc3fd9a5f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56e5dff40f13467fba67f990802c6558'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:31.819629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1240/2497] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:53:31 2022] DELETE /identity/v3/users/56e5dff40f13467fba67f990802c6558 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:32.351023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd67e17-7dbc-4d60-9201-40efec0b5aad service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:32.351544 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5fd67e17-7dbc-4d60-9201-40efec0b5aad service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:32.389983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd67e17-7dbc-4d60-9201-40efec0b5aad service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:32.506074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:32.506949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:32.507226 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:32.507497 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] PATH_INFO: `/v3/projects/9eb17174a4d1402488588f86f5bec0bc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:32.509549 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb17174a4d1402488588f86f5bec0bc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:32.511514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:32.647481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:32.668391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb17174a4d1402488588f86f5bec0bc', 'request_id': 'req-b75b4d60-38a4-4937-b726-ecd231104604'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:32.767723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb17174a4d1402488588f86f5bec0bc', 'request_id': 'req-b75b4d60-38a4-4937-b726-ecd231104604'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:32.797904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb17174a4d1402488588f86f5bec0bc', 'request_id': 'req-b75b4d60-38a4-4937-b726-ecd231104604'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:32.826913 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] The token cache is being invalidate because project 9eb17174a4d1402488588f86f5bec0bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:53:32.827212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:32.829205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b75b4d60-38a4-4937-b726-ecd231104604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:32.833054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1258/2498] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:53:32 2022] DELETE /identity/v3/projects/9eb17174a4d1402488588f86f5bec0bc => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:32.838853 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e583b3-e51b-40fd-b2f6-74b6680f6a8e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:32.839351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e583b3-e51b-40fd-b2f6-74b6680f6a8e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:32.839463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e583b3-e51b-40fd-b2f6-74b6680f6a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:32.867014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e6e583b3-e51b-40fd-b2f6-74b6680f6a8e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:32.868476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e6e583b3-e51b-40fd-b2f6-74b6680f6a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:33.016111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1241/2499] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:53:33.038511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ef866-0d3b-4a4a-a6d6-897a6dad5f1d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:33.038969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ef866-0d3b-4a4a-a6d6-897a6dad5f1d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:33.039381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ef866-0d3b-4a4a-a6d6-897a6dad5f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:33.093359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e90ef866-0d3b-4a4a-a6d6-897a6dad5f1d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:33.099428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e90ef866-0d3b-4a4a-a6d6-897a6dad5f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:33.140626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1259/2500] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:33.141151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e583b3-e51b-40fd-b2f6-74b6680f6a8e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:33.155503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e583b3-e51b-40fd-b2f6-74b6680f6a8e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:33.162798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8761901-7bb5-4525-ab6b-5f3f243619d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:33.164085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8761901-7bb5-4525-ab6b-5f3f243619d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:33.164523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8761901-7bb5-4525-ab6b-5f3f243619d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:33.165064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8761901-7bb5-4525-ab6b-5f3f243619d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:33.166328 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8761901-7bb5-4525-ab6b-5f3f243619d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:53:33.174406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8761901-7bb5-4525-ab6b-5f3f243619d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:33.176648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8761901-7bb5-4525-ab6b-5f3f243619d3 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:33.196684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1242/2501] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:53:33.201856 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ef866-0d3b-4a4a-a6d6-897a6dad5f1d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:33.216477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ef866-0d3b-4a4a-a6d6-897a6dad5f1d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:33.244385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e43294-58c0-4d22-9100-609e61e949cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:33.244385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e43294-58c0-4d22-9100-609e61e949cd admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:33.244385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e43294-58c0-4d22-9100-609e61e949cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:33.244385 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e43294-58c0-4d22-9100-609e61e949cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:33.245106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e43294-58c0-4d22-9100-609e61e949cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:33.245106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e43294-58c0-4d22-9100-609e61e949cd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:33.484622 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8761901-7bb5-4525-ab6b-5f3f243619d3 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:33.494190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1260/2502] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:53:33 2022] POST /identity/v3/projects => generated 361 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:33.502159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8761901-7bb5-4525-ab6b-5f3f243619d3 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:33.673681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f97d7-67e2-44e7-b9a5-0a0180234c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:33.678497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f97d7-67e2-44e7-b9a5-0a0180234c69 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:33.678497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f97d7-67e2-44e7-b9a5-0a0180234c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:33.678497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f97d7-67e2-44e7-b9a5-0a0180234c69 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:33.678497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f97d7-67e2-44e7-b9a5-0a0180234c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:33.678497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f97d7-67e2-44e7-b9a5-0a0180234c69 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:33.774551 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1243/2503] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:53:33 2022] POST /identity/v3/users => generated 281 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:33.779118 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e43294-58c0-4d22-9100-609e61e949cd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:33.795518 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e43294-58c0-4d22-9100-609e61e949cd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:33.803507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102546f-0f92-49dc-bf70-52d6ff473f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:33.804654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102546f-0f92-49dc-bf70-52d6ff473f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:33.804654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102546f-0f92-49dc-bf70-52d6ff473f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:33.804849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102546f-0f92-49dc-bf70-52d6ff473f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:33.805815 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102546f-0f92-49dc-bf70-52d6ff473f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:33.809119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102546f-0f92-49dc-bf70-52d6ff473f00 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:33.833847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1261/2504] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:33.838535 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f97d7-67e2-44e7-b9a5-0a0180234c69 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:33.850610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f97d7-67e2-44e7-b9a5-0a0180234c69 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:33.856623 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e2ef7-0daf-49e9-bad3-dbd727b752a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:33.857477 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e2ef7-0daf-49e9-bad3-dbd727b752a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:33.858353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e2ef7-0daf-49e9-bad3-dbd727b752a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:33.859191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e2ef7-0daf-49e9-bad3-dbd727b752a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:33.860059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e2ef7-0daf-49e9-bad3-dbd727b752a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:33.866398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e2ef7-0daf-49e9-bad3-dbd727b752a4 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:33.908580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102546f-0f92-49dc-bf70-52d6ff473f00 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:33.916786 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1244/2505] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:33.920841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102546f-0f92-49dc-bf70-52d6ff473f00 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:33.933891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712a6bb-15ab-45ed-93ee-7d7cc39890bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:33.935095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712a6bb-15ab-45ed-93ee-7d7cc39890bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:33.935621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712a6bb-15ab-45ed-93ee-7d7cc39890bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:33.936381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712a6bb-15ab-45ed-93ee-7d7cc39890bd admin admin] PATH_INFO: `/v3/projects/3e478ffe2a634d1f8ac321cdd0848588/users/2782c89ac83b487ba06a57f0cf35ae70/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:33.984039 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712a6bb-15ab-45ed-93ee-7d7cc39890bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e478ffe2a634d1f8ac321cdd0848588, user_id=2782c89ac83b487ba06a57f0cf35ae70, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:33.987129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712a6bb-15ab-45ed-93ee-7d7cc39890bd admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:34.094637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1262/2506] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:33 2022] PUT /identity/v3/projects/3e478ffe2a634d1f8ac321cdd0848588/users/2782c89ac83b487ba06a57f0cf35ae70/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:34.096842 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26f3bb-720c-4bb0-931b-592a0eec58b2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:34.097163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26f3bb-720c-4bb0-931b-592a0eec58b2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:34.097442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26f3bb-720c-4bb0-931b-592a0eec58b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:34.121422 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:34.135056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-eb26f3bb-720c-4bb0-931b-592a0eec58b2 None None] MFA Rules not processed for user `2782c89ac83b487ba06a57f0cf35ae70`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:34.136317 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-eb26f3bb-720c-4bb0-931b-592a0eec58b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:34.310808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1245/2507] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:34.343841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712a6bb-15ab-45ed-93ee-7d7cc39890bd admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:34.345616 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9712a6bb-15ab-45ed-93ee-7d7cc39890bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:34.470320 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712a6bb-15ab-45ed-93ee-7d7cc39890bd admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:34.483815 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525e7c1-416b-4d4e-9a01-9d978d5e193f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:34.484664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525e7c1-416b-4d4e-9a01-9d978d5e193f service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:34.484994 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525e7c1-416b-4d4e-9a01-9d978d5e193f service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:34.485282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525e7c1-416b-4d4e-9a01-9d978d5e193f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:34.505416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525e7c1-416b-4d4e-9a01-9d978d5e193f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:34.507011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525e7c1-416b-4d4e-9a01-9d978d5e193f service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:34.522819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1263/2508] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:34.539273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26f3bb-720c-4bb0-931b-592a0eec58b2 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:34.678119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26f3bb-720c-4bb0-931b-592a0eec58b2 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:34.683951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:34.685312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:34.685615 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:34.685927 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] PATH_INFO: `/v3/users/2782c89ac83b487ba06a57f0cf35ae70` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:34.687228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2782c89ac83b487ba06a57f0cf35ae70)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:34.691527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:34.907556 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2782c89ac83b487ba06a57f0cf35ae70'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:34.921252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2782c89ac83b487ba06a57f0cf35ae70'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:34.950601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2782c89ac83b487ba06a57f0cf35ae70'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:35.011567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2782c89ac83b487ba06a57f0cf35ae70'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:35.015274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2782c89ac83b487ba06a57f0cf35ae70'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:35.029491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1246/2509] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:53:34 2022] DELETE /identity/v3/users/2782c89ac83b487ba06a57f0cf35ae70 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:35.036749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ee67f-d4bf-4071-b60d-5ca13d250b90 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:35.037074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ee67f-d4bf-4071-b60d-5ca13d250b90 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:35.037551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ee67f-d4bf-4071-b60d-5ca13d250b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:35.077362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-406ee67f-d4bf-4071-b60d-5ca13d250b90 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:35.078613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-406ee67f-d4bf-4071-b60d-5ca13d250b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:35.222625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:35.236801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1264/2510] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:35.248945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:35.255576 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:35.300028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:35.369286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b153b-011b-4a67-b06f-59100894bcc2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:35.377899 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0f89c-7b2d-403b-9ec4-bfa8b01f9e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:35.377899 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0f89c-7b2d-403b-9ec4-bfa8b01f9e22 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:35.377899 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0f89c-7b2d-403b-9ec4-bfa8b01f9e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:35.377899 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0f89c-7b2d-403b-9ec4-bfa8b01f9e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:35.401205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0f89c-7b2d-403b-9ec4-bfa8b01f9e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:35.406964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0f89c-7b2d-403b-9ec4-bfa8b01f9e22 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:35.420706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1247/2511] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:36.120827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ee67f-d4bf-4071-b60d-5ca13d250b90 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:36.124709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-406ee67f-d4bf-4071-b60d-5ca13d250b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:36.169220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ee67f-d4bf-4071-b60d-5ca13d250b90 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:36.175894 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:36.177204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:36.177697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:36.178095 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] PATH_INFO: `/v3/projects/3e478ffe2a634d1f8ac321cdd0848588` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:36.179795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e478ffe2a634d1f8ac321cdd0848588)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:36.183334 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:36.337464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e478ffe2a634d1f8ac321cdd0848588', 'request_id': 'req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:36.406729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e478ffe2a634d1f8ac321cdd0848588', 'request_id': 'req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:36.414223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e478ffe2a634d1f8ac321cdd0848588', 'request_id': 'req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:36.430204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] The token cache is being invalidate because project 3e478ffe2a634d1f8ac321cdd0848588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:53:36.430511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:36.431207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1676c3d7-02b6-4abb-acc3-6ff3b1d5f693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:36.433004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1265/2512] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:53:36 2022] DELETE /identity/v3/projects/3e478ffe2a634d1f8ac321cdd0848588 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:36.438454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8271d-ec9b-469c-b19a-1747e3a077b6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:36.439289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8271d-ec9b-469c-b19a-1747e3a077b6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:36.439289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8271d-ec9b-469c-b19a-1747e3a077b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:36.465179 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c3d8271d-ec9b-469c-b19a-1747e3a077b6 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:36.466837 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c3d8271d-ec9b-469c-b19a-1747e3a077b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:36.594580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1248/2513] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:53:36.602370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c659a21-6756-4992-83b9-5585760bbe9c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:36.602664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c659a21-6756-4992-83b9-5585760bbe9c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:36.602944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c659a21-6756-4992-83b9-5585760bbe9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:36.627599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3c659a21-6756-4992-83b9-5585760bbe9c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:36.628826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3c659a21-6756-4992-83b9-5585760bbe9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:36.636967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1266/2514] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:36.642097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8271d-ec9b-469c-b19a-1747e3a077b6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:36.653460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8271d-ec9b-469c-b19a-1747e3a077b6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:36.660988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c122b-a9cd-4014-a116-d600f228c4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:36.661595 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c122b-a9cd-4014-a116-d600f228c4df admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:36.661884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c122b-a9cd-4014-a116-d600f228c4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:36.662163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c122b-a9cd-4014-a116-d600f228c4df admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:36.663135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c122b-a9cd-4014-a116-d600f228c4df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:53:36.663452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c122b-a9cd-4014-a116-d600f228c4df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:36.665112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c122b-a9cd-4014-a116-d600f228c4df admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:36.689881 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1249/2515] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:53:36.694173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c659a21-6756-4992-83b9-5585760bbe9c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:36.709486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c659a21-6756-4992-83b9-5585760bbe9c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:36.716796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad139a67-f6df-478e-8b42-72ea4aa37a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:36.717912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad139a67-f6df-478e-8b42-72ea4aa37a0c admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:36.718198 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad139a67-f6df-478e-8b42-72ea4aa37a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:36.718471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad139a67-f6df-478e-8b42-72ea4aa37a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:36.719180 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad139a67-f6df-478e-8b42-72ea4aa37a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:36.721121 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad139a67-f6df-478e-8b42-72ea4aa37a0c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:36.865514 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1267/2516] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:53:36 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:36.866872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c122b-a9cd-4014-a116-d600f228c4df admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:36.878958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c122b-a9cd-4014-a116-d600f228c4df admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:36.966848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef9734-c5ab-48e7-9ded-0ab8c4020020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:36.967660 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef9734-c5ab-48e7-9ded-0ab8c4020020 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:36.967950 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef9734-c5ab-48e7-9ded-0ab8c4020020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:36.968235 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef9734-c5ab-48e7-9ded-0ab8c4020020 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:36.969164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef9734-c5ab-48e7-9ded-0ab8c4020020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:36.971287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef9734-c5ab-48e7-9ded-0ab8c4020020 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:37.079046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1250/2517] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:53:36 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:37.080074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad139a67-f6df-478e-8b42-72ea4aa37a0c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:37.100532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad139a67-f6df-478e-8b42-72ea4aa37a0c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:37.105365 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53ed41-593b-443d-a86a-c4c20c5a59ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:37.106075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53ed41-593b-443d-a86a-c4c20c5a59ca admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:37.106361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53ed41-593b-443d-a86a-c4c20c5a59ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:37.106659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53ed41-593b-443d-a86a-c4c20c5a59ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:37.107411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d53ed41-593b-443d-a86a-c4c20c5a59ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:37.110556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d53ed41-593b-443d-a86a-c4c20c5a59ca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:37.125178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1268/2518] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:37.131385 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef9734-c5ab-48e7-9ded-0ab8c4020020 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:37.142857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef9734-c5ab-48e7-9ded-0ab8c4020020 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:37.166080 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110d08d-54cf-4274-9f5a-21d48619cb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:37.166887 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110d08d-54cf-4274-9f5a-21d48619cb3f admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:37.167173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110d08d-54cf-4274-9f5a-21d48619cb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:37.167475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110d08d-54cf-4274-9f5a-21d48619cb3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:37.168425 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110d08d-54cf-4274-9f5a-21d48619cb3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:37.170670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110d08d-54cf-4274-9f5a-21d48619cb3f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:37.194051 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1251/2519] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:53:37.198196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53ed41-593b-443d-a86a-c4c20c5a59ca admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:37.210921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53ed41-593b-443d-a86a-c4c20c5a59ca admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:37.225428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888864e-5686-495c-9b4b-efa2cc27a6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:37.225428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888864e-5686-495c-9b4b-efa2cc27a6fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:37.225428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888864e-5686-495c-9b4b-efa2cc27a6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:37.225428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888864e-5686-495c-9b4b-efa2cc27a6fb admin admin] PATH_INFO: `/v3/projects/d05991fcf128487092e4e50617da2d74/users/27b60c389fff45f08e8a4b596920fcc6/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:37.259945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888864e-5686-495c-9b4b-efa2cc27a6fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05991fcf128487092e4e50617da2d74, user_id=27b60c389fff45f08e8a4b596920fcc6, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:37.262703 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888864e-5686-495c-9b4b-efa2cc27a6fb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:37.380024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1269/2520] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:37 2022] PUT /identity/v3/projects/d05991fcf128487092e4e50617da2d74/users/27b60c389fff45f08e8a4b596920fcc6/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:37.382010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c7896-d592-4179-8efe-9655d208fe72 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:37.382507 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c7896-d592-4179-8efe-9655d208fe72 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:37.383017 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c7896-d592-4179-8efe-9655d208fe72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:37.437047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-769c7896-d592-4179-8efe-9655d208fe72 None None] MFA Rules not processed for user `27b60c389fff45f08e8a4b596920fcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:37.438679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-769c7896-d592-4179-8efe-9655d208fe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:37.455465 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:37.525781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1252/2521] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:37.530324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888864e-5686-495c-9b4b-efa2cc27a6fb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:37.615962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888864e-5686-495c-9b4b-efa2cc27a6fb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:37.622504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350f6bb-f3e4-4f2f-9922-3df65a6f7fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:37.623044 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350f6bb-f3e4-4f2f-9922-3df65a6f7fff admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:37.623891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350f6bb-f3e4-4f2f-9922-3df65a6f7fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:37.624170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350f6bb-f3e4-4f2f-9922-3df65a6f7fff admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:37.624893 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350f6bb-f3e4-4f2f-9922-3df65a6f7fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:37.626859 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350f6bb-f3e4-4f2f-9922-3df65a6f7fff admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:37.741460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:37.755070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1270/2522] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:53:37 2022] POST /identity/v3/projects => generated 377 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:53:37.759168 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c7896-d592-4179-8efe-9655d208fe72 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:37.772299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c7896-d592-4179-8efe-9655d208fe72 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:37.860491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a7786-ded5-41f5-952b-f5524522cc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:37.861420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a7786-ded5-41f5-952b-f5524522cc3b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:37.861708 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a7786-ded5-41f5-952b-f5524522cc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:37.861986 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a7786-ded5-41f5-952b-f5524522cc3b admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:37.862730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a7786-ded5-41f5-952b-f5524522cc3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:37.864752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a7786-ded5-41f5-952b-f5524522cc3b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:37.954285 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1253/2523] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:53:37 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:37.958361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350f6bb-f3e4-4f2f-9922-3df65a6f7fff admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:37.971554 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350f6bb-f3e4-4f2f-9922-3df65a6f7fff admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:37.977223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4004d9-97b9-4efd-8916-fb44d4e7fa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:37.978023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4004d9-97b9-4efd-8916-fb44d4e7fa7d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:37.978318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4004d9-97b9-4efd-8916-fb44d4e7fa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:37.978596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4004d9-97b9-4efd-8916-fb44d4e7fa7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:37.979545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4004d9-97b9-4efd-8916-fb44d4e7fa7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:37.981289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4004d9-97b9-4efd-8916-fb44d4e7fa7d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:37.995775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1271/2524] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:37.999676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a7786-ded5-41f5-952b-f5524522cc3b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:38.011803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a7786-ded5-41f5-952b-f5524522cc3b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:38.017722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6503a3-634b-4a80-abb7-4211acdc3da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:38.018817 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6503a3-634b-4a80-abb7-4211acdc3da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:38.019321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6503a3-634b-4a80-abb7-4211acdc3da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:38.019722 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6503a3-634b-4a80-abb7-4211acdc3da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:38.020574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6503a3-634b-4a80-abb7-4211acdc3da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:38.022899 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6503a3-634b-4a80-abb7-4211acdc3da8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:38.037103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1254/2525] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:38.040990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4004d9-97b9-4efd-8916-fb44d4e7fa7d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:38.053397 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4004d9-97b9-4efd-8916-fb44d4e7fa7d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:38.058216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdae00-a98d-4d0c-8909-b7ae2e794cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:38.059205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdae00-a98d-4d0c-8909-b7ae2e794cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:38.059511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdae00-a98d-4d0c-8909-b7ae2e794cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:38.060074 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdae00-a98d-4d0c-8909-b7ae2e794cd2 admin admin] PATH_INFO: `/v3/projects/ce1ad08c4c074b8b91d91dbb3f5bb199/users/0d4ca269018541a788f1276e3bf06cb9/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:38.079555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdae00-a98d-4d0c-8909-b7ae2e794cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1ad08c4c074b8b91d91dbb3f5bb199, user_id=0d4ca269018541a788f1276e3bf06cb9, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:38.082933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdae00-a98d-4d0c-8909-b7ae2e794cd2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:38.227211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1272/2526] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:38 2022] PUT /identity/v3/projects/ce1ad08c4c074b8b91d91dbb3f5bb199/users/0d4ca269018541a788f1276e3bf06cb9/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:53:38.237159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f268910-711a-4d28-bbcf-e58b9987eb18 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:38.237682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f268910-711a-4d28-bbcf-e58b9987eb18 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:38.238162 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f268910-711a-4d28-bbcf-e58b9987eb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:38.277880 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-1f268910-711a-4d28-bbcf-e58b9987eb18 None None] MFA Rules not processed for user `0d4ca269018541a788f1276e3bf06cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:38.279199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1f268910-711a-4d28-bbcf-e58b9987eb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:38.371751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1255/2527] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:38.376416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdae00-a98d-4d0c-8909-b7ae2e794cd2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:38.475949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdae00-a98d-4d0c-8909-b7ae2e794cd2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:38.487637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ba83c-76db-4843-8473-778e6964449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:38.488261 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ba83c-76db-4843-8473-778e6964449f admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:38.488542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ba83c-76db-4843-8473-778e6964449f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:38.488993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ba83c-76db-4843-8473-778e6964449f admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:38.489647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ba83c-76db-4843-8473-778e6964449f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:38.492104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ba83c-76db-4843-8473-778e6964449f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:38.705996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1273/2528] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:53:38 2022] POST /identity/v3/projects => generated 377 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:38.726305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f268910-711a-4d28-bbcf-e58b9987eb18 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:38.738053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f268910-711a-4d28-bbcf-e58b9987eb18 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:38.759935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:38.798956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54246c67-6ba6-4ecb-b0e4-da1406191cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:38.799638 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54246c67-6ba6-4ecb-b0e4-da1406191cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:38.799924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54246c67-6ba6-4ecb-b0e4-da1406191cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:38.800199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54246c67-6ba6-4ecb-b0e4-da1406191cb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:38.800941 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54246c67-6ba6-4ecb-b0e4-da1406191cb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:38.802910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54246c67-6ba6-4ecb-b0e4-da1406191cb1 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:39.261637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1256/2529] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:53:38 2022] POST /identity/v3/users => generated 295 bytes in 536 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 16:53:39.262702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ba83c-76db-4843-8473-778e6964449f admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:39.274277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ba83c-76db-4843-8473-778e6964449f admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:39.297337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98521f-b99c-49aa-9946-f93efa5ddd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:39.297337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98521f-b99c-49aa-9946-f93efa5ddd26 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:39.297337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98521f-b99c-49aa-9946-f93efa5ddd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:39.297337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98521f-b99c-49aa-9946-f93efa5ddd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:39.297710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98521f-b99c-49aa-9946-f93efa5ddd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:39.297710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98521f-b99c-49aa-9946-f93efa5ddd26 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:39.309163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1274/2530] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 16:53:39.315815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54246c67-6ba6-4ecb-b0e4-da1406191cb1 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:39.329262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54246c67-6ba6-4ecb-b0e4-da1406191cb1 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:39.349453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac14a10-8f53-4c8f-b23b-6697cbe9ee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:39.349453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac14a10-8f53-4c8f-b23b-6697cbe9ee0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:39.349453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac14a10-8f53-4c8f-b23b-6697cbe9ee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:39.349453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac14a10-8f53-4c8f-b23b-6697cbe9ee0d admin admin] PATH_INFO: `/v3/projects/f57496db5e7d442f83d6f5f171ede998/users/14c6b00750a940a2a9d779c2eeee7a97/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:39.372513 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac14a10-8f53-4c8f-b23b-6697cbe9ee0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f57496db5e7d442f83d6f5f171ede998, user_id=14c6b00750a940a2a9d779c2eeee7a97, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:39.378247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac14a10-8f53-4c8f-b23b-6697cbe9ee0d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:39.458796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1257/2531] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:39 2022] PUT /identity/v3/projects/f57496db5e7d442f83d6f5f171ede998/users/14c6b00750a940a2a9d779c2eeee7a97/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:39.462445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98521f-b99c-49aa-9946-f93efa5ddd26 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:39.474355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98521f-b99c-49aa-9946-f93efa5ddd26 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:39.547442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747aaa7-b044-4f3a-8389-b12a1dae054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:39.548153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747aaa7-b044-4f3a-8389-b12a1dae054a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:39.548512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747aaa7-b044-4f3a-8389-b12a1dae054a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:39.548863 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747aaa7-b044-4f3a-8389-b12a1dae054a admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:39.549769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747aaa7-b044-4f3a-8389-b12a1dae054a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:39.551823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747aaa7-b044-4f3a-8389-b12a1dae054a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:39.566034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1275/2532] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:39.570234 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac14a10-8f53-4c8f-b23b-6697cbe9ee0d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:39.584160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac14a10-8f53-4c8f-b23b-6697cbe9ee0d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:39.593795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0674f6-aa44-4302-9595-c686b9972171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:39.594705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0674f6-aa44-4302-9595-c686b9972171 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:39.594994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0674f6-aa44-4302-9595-c686b9972171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:39.595368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0674f6-aa44-4302-9595-c686b9972171 admin admin] PATH_INFO: `/v3/projects/f57496db5e7d442f83d6f5f171ede998/users/14c6b00750a940a2a9d779c2eeee7a97/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:39.598151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0674f6-aa44-4302-9595-c686b9972171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f57496db5e7d442f83d6f5f171ede998, user_id=14c6b00750a940a2a9d779c2eeee7a97, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:39.600872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0674f6-aa44-4302-9595-c686b9972171 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:39.618457 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1258/2533] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:39 2022] PUT /identity/v3/projects/f57496db5e7d442f83d6f5f171ede998/users/14c6b00750a940a2a9d779c2eeee7a97/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:39.625667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a1a7e-d155-4820-8c9f-430d8236a34e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:39.626522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a1a7e-d155-4820-8c9f-430d8236a34e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:39.627149 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a1a7e-d155-4820-8c9f-430d8236a34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:39.662794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6b4a1a7e-d155-4820-8c9f-430d8236a34e None None] MFA Rules not processed for user `14c6b00750a940a2a9d779c2eeee7a97`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:39.665247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6b4a1a7e-d155-4820-8c9f-430d8236a34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:39.688027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:39.754268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1276/2534] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:39.762270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0674f6-aa44-4302-9595-c686b9972171 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:39.764160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4b0674f6-aa44-4302-9595-c686b9972171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:39.836451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0674f6-aa44-4302-9595-c686b9972171 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:39.841197 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a1203-2b81-4f16-acf1-d572a302d162 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:39.841967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a1203-2b81-4f16-acf1-d572a302d162 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:39.843511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a1203-2b81-4f16-acf1-d572a302d162 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:39.843511 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a1203-2b81-4f16-acf1-d572a302d162 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:39.913519 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a1203-2b81-4f16-acf1-d572a302d162 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:39.915219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a1203-2b81-4f16-acf1-d572a302d162 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:39.928189 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1259/2535] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:42.511216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a1a7e-d155-4820-8c9f-430d8236a34e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:42.512642 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6b4a1a7e-d155-4820-8c9f-430d8236a34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:42.529830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a1a7e-d155-4820-8c9f-430d8236a34e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:42.535946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef85104-bc56-42ee-bc2f-8782318703a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:42.536909 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef85104-bc56-42ee-bc2f-8782318703a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:42.537528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef85104-bc56-42ee-bc2f-8782318703a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:42.538347 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef85104-bc56-42ee-bc2f-8782318703a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:42.638838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef85104-bc56-42ee-bc2f-8782318703a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:42.642077 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef85104-bc56-42ee-bc2f-8782318703a3 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:42.670806 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1277/2536] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:43.615873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f50bd-ced5-476c-8984-f02d7d7ea53f None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:43.618928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f50bd-ced5-476c-8984-f02d7d7ea53f None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:43.618928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f50bd-ced5-476c-8984-f02d7d7ea53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:43.634891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-2f9f50bd-ced5-476c-8984-f02d7d7ea53f None None] MFA Rules not processed for user `ab4d3667826148d390eda90bb8baf081`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:43.638194 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2f9f50bd-ced5-476c-8984-f02d7d7ea53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:43.664855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:43.700924 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1260/2537] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:44.531107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef85104-bc56-42ee-bc2f-8782318703a3 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:44.539986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef85104-bc56-42ee-bc2f-8782318703a3 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:44.545356 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66d462-01e9-4bec-b65a-4ff7163b3889 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:44.546208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66d462-01e9-4bec-b65a-4ff7163b3889 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:44.546682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66d462-01e9-4bec-b65a-4ff7163b3889 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:44.546966 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66d462-01e9-4bec-b65a-4ff7163b3889 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:44.557988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66d462-01e9-4bec-b65a-4ff7163b3889 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:44.571464 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66d462-01e9-4bec-b65a-4ff7163b3889 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:44.573802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1278/2538] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:53:47.633457 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf07eb4-9a08-4c55-a5a0-4031be2e25c6 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:47.633918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf07eb4-9a08-4c55-a5a0-4031be2e25c6 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:47.634131 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf07eb4-9a08-4c55-a5a0-4031be2e25c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:47.654389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4bf07eb4-9a08-4c55-a5a0-4031be2e25c6 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:47.656068 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4bf07eb4-9a08-4c55-a5a0-4031be2e25c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:47.764043 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1261/2539] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:53:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:47.777736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66d462-01e9-4bec-b65a-4ff7163b3889 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:47.781299 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9b66d462-01e9-4bec-b65a-4ff7163b3889 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:47.884233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66d462-01e9-4bec-b65a-4ff7163b3889 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:47.895600 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfcbac-25b7-4c87-9358-bbc4fb826793 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:47.897857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfcbac-25b7-4c87-9358-bbc4fb826793 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:47.897857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfcbac-25b7-4c87-9358-bbc4fb826793 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:47.897857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfcbac-25b7-4c87-9358-bbc4fb826793 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:47.910608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfcbac-25b7-4c87-9358-bbc4fb826793 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:47.912281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfcbac-25b7-4c87-9358-bbc4fb826793 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:47.925654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1279/2540] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:53:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:53:49.403228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf07eb4-9a08-4c55-a5a0-4031be2e25c6 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:49.510162 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf07eb4-9a08-4c55-a5a0-4031be2e25c6 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:49.515582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:49.516466 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:49.516762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:49.517074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] PATH_INFO: `/v3/users/27b60c389fff45f08e8a4b596920fcc6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:49.518326 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27b60c389fff45f08e8a4b596920fcc6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:49.520391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:49.750899 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27b60c389fff45f08e8a4b596920fcc6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:49.760686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27b60c389fff45f08e8a4b596920fcc6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:49.777355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27b60c389fff45f08e8a4b596920fcc6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:49.909958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27b60c389fff45f08e8a4b596920fcc6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:49.910641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27b60c389fff45f08e8a4b596920fcc6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:49.913724 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:49.919523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1262/2541] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:53:49 2022] DELETE /identity/v3/users/27b60c389fff45f08e8a4b596920fcc6 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:49.923055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3c539-5489-496b-9083-b6bb70b965f3 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:49.924051 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3c539-5489-496b-9083-b6bb70b965f3 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:49.924552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3c539-5489-496b-9083-b6bb70b965f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:49.957268 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-eda3c539-5489-496b-9083-b6bb70b965f3 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:49.958556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eda3c539-5489-496b-9083-b6bb70b965f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:50.002974 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:50.034721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1280/2542] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:50.044320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:50.046203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:50.114803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390379d4-a6fa-47b8-bef3-2223ed1cf000 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:50.118669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762ac76-af33-4d0d-b943-928ed871ca18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:50.119322 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762ac76-af33-4d0d-b943-928ed871ca18 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:50.119637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762ac76-af33-4d0d-b943-928ed871ca18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:50.119933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762ac76-af33-4d0d-b943-928ed871ca18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:50.132562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762ac76-af33-4d0d-b943-928ed871ca18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:50.134601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762ac76-af33-4d0d-b943-928ed871ca18 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:50.161031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1263/2543] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:53:51.126671 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3c539-5489-496b-9083-b6bb70b965f3 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:51.127238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eda3c539-5489-496b-9083-b6bb70b965f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:51.150191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3c539-5489-496b-9083-b6bb70b965f3 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:51.163269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:51.164813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:51.165070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:51.165346 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] PATH_INFO: `/v3/projects/d05991fcf128487092e4e50617da2d74` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:51.166769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05991fcf128487092e4e50617da2d74)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:51.169511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:51.335481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05991fcf128487092e4e50617da2d74', 'request_id': 'req-fb8a3789-40a9-4ea0-91e2-21c67202020d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:51.422594 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05991fcf128487092e4e50617da2d74', 'request_id': 'req-fb8a3789-40a9-4ea0-91e2-21c67202020d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:51.430199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05991fcf128487092e4e50617da2d74', 'request_id': 'req-fb8a3789-40a9-4ea0-91e2-21c67202020d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:51.442432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] The token cache is being invalidate because project d05991fcf128487092e4e50617da2d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:53:51.442738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:51.443522 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:51.445184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1281/2544] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:53:51 2022] DELETE /identity/v3/projects/d05991fcf128487092e4e50617da2d74 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:51.449716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762ac76-af33-4d0d-b943-928ed871ca18 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:51.461109 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762ac76-af33-4d0d-b943-928ed871ca18 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:51.532405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:51.533329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:51.533629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:51.533912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] PATH_INFO: `/v3/users/0d4ca269018541a788f1276e3bf06cb9` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:51.535146 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d4ca269018541a788f1276e3bf06cb9)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:51.537146 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:51.716520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d4ca269018541a788f1276e3bf06cb9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:51.724964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d4ca269018541a788f1276e3bf06cb9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:51.746140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d4ca269018541a788f1276e3bf06cb9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:51.822430 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d4ca269018541a788f1276e3bf06cb9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:51.823062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d4ca269018541a788f1276e3bf06cb9'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:51.833798 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1264/2545] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:53:51 2022] DELETE /identity/v3/users/0d4ca269018541a788f1276e3bf06cb9 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:52.403232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:52.405460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:52.422367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a3789-40a9-4ea0-91e2-21c67202020d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:52.503170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:52.503170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:52.503170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:52.503170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] PATH_INFO: `/v3/projects/ce1ad08c4c074b8b91d91dbb3f5bb199` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:52.503897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce1ad08c4c074b8b91d91dbb3f5bb199)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:52.503897 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:52.961126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce1ad08c4c074b8b91d91dbb3f5bb199', 'request_id': 'req-1b8239dc-1754-4af1-b110-f47421074d32'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:53.050653 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce1ad08c4c074b8b91d91dbb3f5bb199', 'request_id': 'req-1b8239dc-1754-4af1-b110-f47421074d32'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:53.053070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:53.057242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce1ad08c4c074b8b91d91dbb3f5bb199', 'request_id': 'req-1b8239dc-1754-4af1-b110-f47421074d32'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:53.069148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] The token cache is being invalidate because project ce1ad08c4c074b8b91d91dbb3f5bb199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:53:53.069486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:53.070116 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:53.071896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1282/2546] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:53:52 2022] DELETE /identity/v3/projects/ce1ad08c4c074b8b91d91dbb3f5bb199 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:53.075942 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:53.077528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:53.114305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f3e21-7de5-4029-9343-d28df1a88def admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:53.136150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:53.182177 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83750639-f4a1-4d09-a678-b899a247627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:53.183228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750639-f4a1-4d09-a678-b899a247627e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:53.183521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750639-f4a1-4d09-a678-b899a247627e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:53.183866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83750639-f4a1-4d09-a678-b899a247627e admin admin] PATH_INFO: `/v3/users/14c6b00750a940a2a9d779c2eeee7a97` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:53.185259 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83750639-f4a1-4d09-a678-b899a247627e admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c6b00750a940a2a9d779c2eeee7a97)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:53.187454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83750639-f4a1-4d09-a678-b899a247627e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:53.420706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-83750639-f4a1-4d09-a678-b899a247627e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c6b00750a940a2a9d779c2eeee7a97'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:53.429090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-83750639-f4a1-4d09-a678-b899a247627e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c6b00750a940a2a9d779c2eeee7a97'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:53.443069 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-83750639-f4a1-4d09-a678-b899a247627e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c6b00750a940a2a9d779c2eeee7a97'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:53.534020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-83750639-f4a1-4d09-a678-b899a247627e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c6b00750a940a2a9d779c2eeee7a97'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:53.549821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-83750639-f4a1-4d09-a678-b899a247627e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c6b00750a940a2a9d779c2eeee7a97'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:53.549821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1265/2547] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:53:53 2022] DELETE /identity/v3/users/14c6b00750a940a2a9d779c2eeee7a97 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:53:54.372043 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:54.373748 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:54.391234 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8239dc-1754-4af1-b110-f47421074d32 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:54.504011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:54.504908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:54.505314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:54.505661 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] PATH_INFO: `/v3/projects/f57496db5e7d442f83d6f5f171ede998` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:54.507525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f57496db5e7d442f83d6f5f171ede998)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:54.509690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:54.651332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f57496db5e7d442f83d6f5f171ede998', 'request_id': 'req-b5019bf5-c317-4f9f-a251-5cb15c33d931'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:54.742242 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f57496db5e7d442f83d6f5f171ede998', 'request_id': 'req-b5019bf5-c317-4f9f-a251-5cb15c33d931'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:54.750666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f57496db5e7d442f83d6f5f171ede998', 'request_id': 'req-b5019bf5-c317-4f9f-a251-5cb15c33d931'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:54.766568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] The token cache is being invalidate because project f57496db5e7d442f83d6f5f171ede998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:53:54.766852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:54.767513 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-b5019bf5-c317-4f9f-a251-5cb15c33d931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:53:54.769425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1283/2548] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:53:54 2022] DELETE /identity/v3/projects/f57496db5e7d442f83d6f5f171ede998 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:54.776303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0638b-3fc7-409f-a8b1-b443a35bac1e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:54.777012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0638b-3fc7-409f-a8b1-b443a35bac1e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:54.777012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0638b-3fc7-409f-a8b1-b443a35bac1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:54.800969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-95d0638b-3fc7-409f-a8b1-b443a35bac1e None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:54.802225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-95d0638b-3fc7-409f-a8b1-b443a35bac1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:54.871327 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1266/2549] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:54.880011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b196c3-75f6-4430-9b3b-0977d17b6090 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:54.880341 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b196c3-75f6-4430-9b3b-0977d17b6090 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:54.880662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b196c3-75f6-4430-9b3b-0977d17b6090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:54.899730 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-40b196c3-75f6-4430-9b3b-0977d17b6090 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:54.900985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-40b196c3-75f6-4430-9b3b-0977d17b6090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:54.910634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1284/2550] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:54.915435 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0638b-3fc7-409f-a8b1-b443a35bac1e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:54.926559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0638b-3fc7-409f-a8b1-b443a35bac1e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:54.934699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6129a39-d2e9-40e0-99c4-3c485d10165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:54.934923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6129a39-d2e9-40e0-99c4-3c485d10165a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:54.935152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6129a39-d2e9-40e0-99c4-3c485d10165a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:54.935424 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6129a39-d2e9-40e0-99c4-3c485d10165a admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:54.936172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6129a39-d2e9-40e0-99c4-3c485d10165a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:53:54.936529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6129a39-d2e9-40e0-99c4-3c485d10165a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:54.938873 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6129a39-d2e9-40e0-99c4-3c485d10165a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:54.961387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1267/2551] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:53:54.965472 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b196c3-75f6-4430-9b3b-0977d17b6090 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:54.975599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b196c3-75f6-4430-9b3b-0977d17b6090 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:54.981016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449bb97-8ed1-46fb-977b-f91dc2446fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:54.982400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449bb97-8ed1-46fb-977b-f91dc2446fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:54.982690 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449bb97-8ed1-46fb-977b-f91dc2446fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:54.982971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449bb97-8ed1-46fb-977b-f91dc2446fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:54.984222 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449bb97-8ed1-46fb-977b-f91dc2446fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:54.986166 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449bb97-8ed1-46fb-977b-f91dc2446fd4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:55.081295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1285/2552] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:53:54 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:55.085879 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6129a39-d2e9-40e0-99c4-3c485d10165a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:55.097539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6129a39-d2e9-40e0-99c4-3c485d10165a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:55.128452 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:55.161019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7ba69-e856-4c64-86ef-a9e70285b914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:55.161717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7ba69-e856-4c64-86ef-a9e70285b914 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:55.161997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7ba69-e856-4c64-86ef-a9e70285b914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:55.162270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7ba69-e856-4c64-86ef-a9e70285b914 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:55.162970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7ba69-e856-4c64-86ef-a9e70285b914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:55.164944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7ba69-e856-4c64-86ef-a9e70285b914 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:55.257172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1268/2553] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:53:55 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:53:55.262350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449bb97-8ed1-46fb-977b-f91dc2446fd4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:55.277202 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449bb97-8ed1-46fb-977b-f91dc2446fd4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:55.283781 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e30bd3-7430-4503-b055-374e355227a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:55.284468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e30bd3-7430-4503-b055-374e355227a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:55.284755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e30bd3-7430-4503-b055-374e355227a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:55.285071 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e30bd3-7430-4503-b055-374e355227a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:55.285815 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e30bd3-7430-4503-b055-374e355227a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:55.287511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e30bd3-7430-4503-b055-374e355227a6 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:55.303041 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1286/2554] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:55.304278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7ba69-e856-4c64-86ef-a9e70285b914 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:55.315688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7ba69-e856-4c64-86ef-a9e70285b914 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:55.320571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2701c2f-54c6-4f70-9e93-a3802c99d695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:55.321280 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2701c2f-54c6-4f70-9e93-a3802c99d695 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:55.321561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2701c2f-54c6-4f70-9e93-a3802c99d695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:55.321836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2701c2f-54c6-4f70-9e93-a3802c99d695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:55.322565 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2701c2f-54c6-4f70-9e93-a3802c99d695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:55.324163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2701c2f-54c6-4f70-9e93-a3802c99d695 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:55.335792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1269/2555] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:53:55.339938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e30bd3-7430-4503-b055-374e355227a6 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:55.357972 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e30bd3-7430-4503-b055-374e355227a6 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:55.363681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020723d-9148-438c-84cf-43833d3a8143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:55.364616 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020723d-9148-438c-84cf-43833d3a8143 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:55.364915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020723d-9148-438c-84cf-43833d3a8143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:55.365195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020723d-9148-438c-84cf-43833d3a8143 admin admin] PATH_INFO: `/v3/projects/5d68154d9fdb40f0848b676c1fed6a50/users/cd69c5111dac492a9177128317f0962a/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:55.381537 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020723d-9148-438c-84cf-43833d3a8143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d68154d9fdb40f0848b676c1fed6a50, user_id=cd69c5111dac492a9177128317f0962a, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:55.384247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020723d-9148-438c-84cf-43833d3a8143 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:55.511597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1287/2556] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:53:55 2022] PUT /identity/v3/projects/5d68154d9fdb40f0848b676c1fed6a50/users/cd69c5111dac492a9177128317f0962a/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:53:55.518156 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469263b6-c380-4e28-9fad-2d123c9ec93b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:55.518588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469263b6-c380-4e28-9fad-2d123c9ec93b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:55.519028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469263b6-c380-4e28-9fad-2d123c9ec93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:55.541212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-469263b6-c380-4e28-9fad-2d123c9ec93b None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:53:55.542580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-469263b6-c380-4e28-9fad-2d123c9ec93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:55.605884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1270/2557] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:53:55.616607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020723d-9148-438c-84cf-43833d3a8143 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:53:55.618375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d020723d-9148-438c-84cf-43833d3a8143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:53:55.635657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:53:55.711462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020723d-9148-438c-84cf-43833d3a8143 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:53:55.715917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d354551-a6a4-4dd9-b33a-7f3d711b7b34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:53:55.718448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d354551-a6a4-4dd9-b33a-7f3d711b7b34 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:53:55.718448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d354551-a6a4-4dd9-b33a-7f3d711b7b34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:53:55.718448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d354551-a6a4-4dd9-b33a-7f3d711b7b34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:53:55.729120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d354551-a6a4-4dd9-b33a-7f3d711b7b34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:53:55.731205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d354551-a6a4-4dd9-b33a-7f3d711b7b34 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:53:55.745565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1288/2558] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:53:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:54:02.435274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8bd79-420a-42b3-a222-ca8595f30891 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:02.435803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8bd79-420a-42b3-a222-ca8595f30891 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:02.435959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8bd79-420a-42b3-a222-ca8595f30891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:02.466814 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8ab8bd79-420a-42b3-a222-ca8595f30891 None None] MFA Rules not processed for user `cd69c5111dac492a9177128317f0962a`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:54:02.468281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8ab8bd79-420a-42b3-a222-ca8595f30891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:02.554046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1271/2559] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:54:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:54:02.558681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d354551-a6a4-4dd9-b33a-7f3d711b7b34 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:02.570622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d354551-a6a4-4dd9-b33a-7f3d711b7b34 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:02.575421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84316f-d853-4f75-ae61-8324dd0ea9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:02.576084 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84316f-d853-4f75-ae61-8324dd0ea9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:02.576370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84316f-d853-4f75-ae61-8324dd0ea9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:02.576647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84316f-d853-4f75-ae61-8324dd0ea9a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:02.577802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84316f-d853-4f75-ae61-8324dd0ea9a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:02.579794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84316f-d853-4f75-ae61-8324dd0ea9a7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:02.704560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1289/2560] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:54:02 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:54:02.709315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8bd79-420a-42b3-a222-ca8595f30891 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:02.721908 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8bd79-420a-42b3-a222-ca8595f30891 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:02.761854 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:54:02.789418 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe86b0b-818e-4ef7-a4dd-4f1ffc0afd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:02.790112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe86b0b-818e-4ef7-a4dd-4f1ffc0afd1e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:02.790397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe86b0b-818e-4ef7-a4dd-4f1ffc0afd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:02.790830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe86b0b-818e-4ef7-a4dd-4f1ffc0afd1e admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:02.791541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe86b0b-818e-4ef7-a4dd-4f1ffc0afd1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:02.793700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe86b0b-818e-4ef7-a4dd-4f1ffc0afd1e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:02.890312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1272/2561] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:54:02 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:54:02.890515 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84316f-d853-4f75-ae61-8324dd0ea9a7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:02.904342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84316f-d853-4f75-ae61-8324dd0ea9a7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:02.912905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16f913-1816-42da-9008-2917adc0a65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:02.914113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f913-1816-42da-9008-2917adc0a65c admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:02.914631 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f913-1816-42da-9008-2917adc0a65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:02.915164 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16f913-1816-42da-9008-2917adc0a65c admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:02.916177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16f913-1816-42da-9008-2917adc0a65c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:02.919076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16f913-1816-42da-9008-2917adc0a65c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:02.934330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1290/2562] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:54:02.938548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe86b0b-818e-4ef7-a4dd-4f1ffc0afd1e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:02.949731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe86b0b-818e-4ef7-a4dd-4f1ffc0afd1e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:02.955084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde7fdb-7602-4e1b-b415-98d3c9680199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:02.956226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde7fdb-7602-4e1b-b415-98d3c9680199 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:02.956508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde7fdb-7602-4e1b-b415-98d3c9680199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:02.956780 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde7fdb-7602-4e1b-b415-98d3c9680199 admin admin] PATH_INFO: `/v3/projects/13009ce503fe4599ae3cf22798bd7f22/users/da3a417ec62d44568f5d4eaaad54f3fa/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:02.973968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde7fdb-7602-4e1b-b415-98d3c9680199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13009ce503fe4599ae3cf22798bd7f22, user_id=da3a417ec62d44568f5d4eaaad54f3fa, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:02.976571 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde7fdb-7602-4e1b-b415-98d3c9680199 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:03.083307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1273/2563] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:54:02 2022] PUT /identity/v3/projects/13009ce503fe4599ae3cf22798bd7f22/users/da3a417ec62d44568f5d4eaaad54f3fa/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:54:03.087235 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16f913-1816-42da-9008-2917adc0a65c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:03.097790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16f913-1816-42da-9008-2917adc0a65c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:03.158616 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82a089-2997-4bb1-857e-af7036326573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:03.159212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82a089-2997-4bb1-857e-af7036326573 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:03.159499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82a089-2997-4bb1-857e-af7036326573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:03.159780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82a089-2997-4bb1-857e-af7036326573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:03.160511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82a089-2997-4bb1-857e-af7036326573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:03.162193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82a089-2997-4bb1-857e-af7036326573 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:03.173223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1291/2564] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:54:03.177402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde7fdb-7602-4e1b-b415-98d3c9680199 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:03.188529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde7fdb-7602-4e1b-b415-98d3c9680199 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:03.193394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bc7a8-547e-43e7-a023-f2d425412846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:03.194283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bc7a8-547e-43e7-a023-f2d425412846 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:03.194570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bc7a8-547e-43e7-a023-f2d425412846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:03.194849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bc7a8-547e-43e7-a023-f2d425412846 admin admin] PATH_INFO: `/v3/projects/13009ce503fe4599ae3cf22798bd7f22/users/da3a417ec62d44568f5d4eaaad54f3fa/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:03.196902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bc7a8-547e-43e7-a023-f2d425412846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13009ce503fe4599ae3cf22798bd7f22, user_id=da3a417ec62d44568f5d4eaaad54f3fa, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:03.199582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bc7a8-547e-43e7-a023-f2d425412846 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:03.210974 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1274/2565] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:54:03 2022] PUT /identity/v3/projects/13009ce503fe4599ae3cf22798bd7f22/users/da3a417ec62d44568f5d4eaaad54f3fa/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:54:09.564119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069996-905f-4f80-bc28-1cfd1bd00780 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:09.565795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069996-905f-4f80-bc28-1cfd1bd00780 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:09.566328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7069996-905f-4f80-bc28-1cfd1bd00780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:09.636212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d7069996-905f-4f80-bc28-1cfd1bd00780 None None] MFA Rules not processed for user `da3a417ec62d44568f5d4eaaad54f3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:54:09.638798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d7069996-905f-4f80-bc28-1cfd1bd00780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:09.731533 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1292/2566] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:54:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:54:09.749220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bc7a8-547e-43e7-a023-f2d425412846 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:09.749220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-239bc7a8-547e-43e7-a023-f2d425412846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:09.865756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bc7a8-547e-43e7-a023-f2d425412846 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:09.872184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead90fe7-25b1-4361-9e12-14e38ad41734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:09.872923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead90fe7-25b1-4361-9e12-14e38ad41734 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:09.873229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead90fe7-25b1-4361-9e12-14e38ad41734 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:09.873514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead90fe7-25b1-4361-9e12-14e38ad41734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:09.959053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead90fe7-25b1-4361-9e12-14e38ad41734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:09.960855 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead90fe7-25b1-4361-9e12-14e38ad41734 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:09.974473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1275/2567] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:54:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:54:19.727626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:54:19.967915 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:54:33.448697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7069996-905f-4f80-bc28-1cfd1bd00780 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:33.507237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7069996-905f-4f80-bc28-1cfd1bd00780 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:33.510974 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752aa0fc-6ce3-49e5-adca-090eec982b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:33.511613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aa0fc-6ce3-49e5-adca-090eec982b6f service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:33.511898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aa0fc-6ce3-49e5-adca-090eec982b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:33.512177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752aa0fc-6ce3-49e5-adca-090eec982b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:33.513998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-752aa0fc-6ce3-49e5-adca-090eec982b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:33.586165 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752aa0fc-6ce3-49e5-adca-090eec982b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:33.587636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752aa0fc-6ce3-49e5-adca-090eec982b6f service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:33.598177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1293/2568] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:54:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:54:37.890148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a155c3-763b-4698-8fa6-3a9299909d52 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:37.890148 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a155c3-763b-4698-8fa6-3a9299909d52 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:37.895170 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a155c3-763b-4698-8fa6-3a9299909d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:37.908268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b7a155c3-763b-4698-8fa6-3a9299909d52 None None] MFA Rules not processed for user `81d0641fc86a43f7a6c772add1e2ae48`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:54:37.909561 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b7a155c3-763b-4698-8fa6-3a9299909d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:38.017614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1276/2569] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:54:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:54:39.186725 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e1cc7-2531-4ddf-8990-083f43ea4493 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:39.187274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e1cc7-2531-4ddf-8990-083f43ea4493 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:39.187274 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e1cc7-2531-4ddf-8990-083f43ea4493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:39.209096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-144e1cc7-2531-4ddf-8990-083f43ea4493 None None] MFA Rules not processed for user `cd69c5111dac492a9177128317f0962a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:54:39.210728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-144e1cc7-2531-4ddf-8990-083f43ea4493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:39.223644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1294/2570] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:54:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:54:46.131259 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580fc2d-99f7-451e-ab0a-fdb3cf1d78dd None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:46.132166 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580fc2d-99f7-451e-ab0a-fdb3cf1d78dd None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:46.132451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580fc2d-99f7-451e-ab0a-fdb3cf1d78dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:46.152196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b580fc2d-99f7-451e-ab0a-fdb3cf1d78dd None None] MFA Rules not processed for user `f2d1f86034b94e289bb1fc754f02f4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:54:46.153718 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b580fc2d-99f7-451e-ab0a-fdb3cf1d78dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:46.215352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1277/2571] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:54:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:54:49.202117 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=53 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:54:56.208203 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=50 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:54:58.483096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e1cc7-2531-4ddf-8990-083f43ea4493 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:58.492932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e1cc7-2531-4ddf-8990-083f43ea4493 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:58.496445 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa268b6-4c4a-4bc6-9e58-f1c2d7c31c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:58.497127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa268b6-4c4a-4bc6-9e58-f1c2d7c31c8e service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:58.497390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa268b6-4c4a-4bc6-9e58-f1c2d7c31c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:58.498311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa268b6-4c4a-4bc6-9e58-f1c2d7c31c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:58.500092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ffa268b6-4c4a-4bc6-9e58-f1c2d7c31c8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:58.517618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa268b6-4c4a-4bc6-9e58-f1c2d7c31c8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:58.522193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa268b6-4c4a-4bc6-9e58-f1c2d7c31c8e service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:54:58.531712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1295/2572] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:54:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:54:59.948596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580fc2d-99f7-451e-ab0a-fdb3cf1d78dd None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:54:59.948596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b580fc2d-99f7-451e-ab0a-fdb3cf1d78dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:59.962977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580fc2d-99f7-451e-ab0a-fdb3cf1d78dd None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:54:59.966921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143a88f-6939-4eb5-880c-a0f9b92cad15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:54:59.968391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143a88f-6939-4eb5-880c-a0f9b92cad15 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:54:59.968682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143a88f-6939-4eb5-880c-a0f9b92cad15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:54:59.968983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143a88f-6939-4eb5-880c-a0f9b92cad15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:54:59.971239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e143a88f-6939-4eb5-880c-a0f9b92cad15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:54:59.988263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143a88f-6939-4eb5-880c-a0f9b92cad15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:54:59.989772 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143a88f-6939-4eb5-880c-a0f9b92cad15 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:00.002841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1278/2573] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:54:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:04.899481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa268b6-4c4a-4bc6-9e58-f1c2d7c31c8e service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:04.908282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa268b6-4c4a-4bc6-9e58-f1c2d7c31c8e service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:04.913730 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf60818-d790-4622-81b4-f0e01ceab273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:04.913730 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf60818-d790-4622-81b4-f0e01ceab273 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:04.913730 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf60818-d790-4622-81b4-f0e01ceab273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:04.913730 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf60818-d790-4622-81b4-f0e01ceab273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:04.922773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf60818-d790-4622-81b4-f0e01ceab273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:04.924215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf60818-d790-4622-81b4-f0e01ceab273 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:04.935778 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1296/2574] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:05.657546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ca559-3d1b-454c-b12f-809e24af617e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:05.658209 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ca559-3d1b-454c-b12f-809e24af617e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:05.658740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ca559-3d1b-454c-b12f-809e24af617e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:05.680026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-346ca559-3d1b-454c-b12f-809e24af617e None None] MFA Rules not processed for user `6890f2e62cfd46999791d4a10a81f95c`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:55:05.681481 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-346ca559-3d1b-454c-b12f-809e24af617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:05.743298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1279/2575] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:55:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:55:13.062458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf60818-d790-4622-81b4-f0e01ceab273 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:13.075865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf60818-d790-4622-81b4-f0e01ceab273 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:13.082097 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82110b5c-d89e-482c-a3b4-201f1de67031 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:13.082934 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82110b5c-d89e-482c-a3b4-201f1de67031 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:13.083229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82110b5c-d89e-482c-a3b4-201f1de67031 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:13.083505 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82110b5c-d89e-482c-a3b4-201f1de67031 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:13.085354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-82110b5c-d89e-482c-a3b4-201f1de67031 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:13.096606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82110b5c-d89e-482c-a3b4-201f1de67031 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:13.098243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82110b5c-d89e-482c-a3b4-201f1de67031 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:13.109760 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1297/2576] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:15.734822 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=31 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:55:23.104272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=18 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:55:41.401243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ca559-3d1b-454c-b12f-809e24af617e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:41.402670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-346ca559-3d1b-454c-b12f-809e24af617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:41.460624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ca559-3d1b-454c-b12f-809e24af617e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:41.464255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8f4f0-5e8d-4dcb-94eb-e145c23a412c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:41.464907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8f4f0-5e8d-4dcb-94eb-e145c23a412c service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:41.465213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8f4f0-5e8d-4dcb-94eb-e145c23a412c service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:41.465491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8f4f0-5e8d-4dcb-94eb-e145c23a412c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:41.467273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0dd8f4f0-5e8d-4dcb-94eb-e145c23a412c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:41.478900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8f4f0-5e8d-4dcb-94eb-e145c23a412c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:41.480331 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8f4f0-5e8d-4dcb-94eb-e145c23a412c service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:41.490147 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1280/2577] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:50.668535 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82110b5c-d89e-482c-a3b4-201f1de67031 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:50.670156 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-82110b5c-d89e-482c-a3b4-201f1de67031 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:50.685005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82110b5c-d89e-482c-a3b4-201f1de67031 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:50.691098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4ff88-70c9-489c-afaa-77bfddcc9b8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:50.691746 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4ff88-70c9-489c-afaa-77bfddcc9b8d service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:50.692033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4ff88-70c9-489c-afaa-77bfddcc9b8d service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:50.692312 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4ff88-70c9-489c-afaa-77bfddcc9b8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:50.694114 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-16b4ff88-70c9-489c-afaa-77bfddcc9b8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:50.707552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4ff88-70c9-489c-afaa-77bfddcc9b8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:50.709111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4ff88-70c9-489c-afaa-77bfddcc9b8d service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:50.720322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1298/2578] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:51.485871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=26 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:55:55.579499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8f4f0-5e8d-4dcb-94eb-e145c23a412c service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:55.581592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0dd8f4f0-5e8d-4dcb-94eb-e145c23a412c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:55.655540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8f4f0-5e8d-4dcb-94eb-e145c23a412c service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:55.661242 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef8583-452d-43ec-9705-31ac0fa419de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:55.661891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef8583-452d-43ec-9705-31ac0fa419de service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:55.662175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef8583-452d-43ec-9705-31ac0fa419de service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:55.662450 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef8583-452d-43ec-9705-31ac0fa419de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:55.665251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-3cef8583-452d-43ec-9705-31ac0fa419de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:55.677524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef8583-452d-43ec-9705-31ac0fa419de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:55.679111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef8583-452d-43ec-9705-31ac0fa419de service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:55.689286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1281/2579] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:55.808641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4ff88-70c9-489c-afaa-77bfddcc9b8d service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:55.808641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-16b4ff88-70c9-489c-afaa-77bfddcc9b8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:55.877424 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4ff88-70c9-489c-afaa-77bfddcc9b8d service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:55.882529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65bce0-4a5e-4649-9133-80601978ce1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:55.883390 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65bce0-4a5e-4649-9133-80601978ce1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:55.883675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65bce0-4a5e-4649-9133-80601978ce1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:55.883950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65bce0-4a5e-4649-9133-80601978ce1f admin admin] PATH_INFO: `/v3/users/81d0641fc86a43f7a6c772add1e2ae48` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:55.885170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65bce0-4a5e-4649-9133-80601978ce1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d0641fc86a43f7a6c772add1e2ae48)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:55.887101 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65bce0-4a5e-4649-9133-80601978ce1f admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:56.102521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b65bce0-4a5e-4649-9133-80601978ce1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d0641fc86a43f7a6c772add1e2ae48'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:56.115141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b65bce0-4a5e-4649-9133-80601978ce1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d0641fc86a43f7a6c772add1e2ae48'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:56.122984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b65bce0-4a5e-4649-9133-80601978ce1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d0641fc86a43f7a6c772add1e2ae48'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:56.191584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b65bce0-4a5e-4649-9133-80601978ce1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d0641fc86a43f7a6c772add1e2ae48'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:56.192282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1b65bce0-4a5e-4649-9133-80601978ce1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d0641fc86a43f7a6c772add1e2ae48'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:56.199237 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1299/2580] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:55:55 2022] DELETE /identity/v3/users/81d0641fc86a43f7a6c772add1e2ae48 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:55:56.449106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef8583-452d-43ec-9705-31ac0fa419de service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:56.461045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef8583-452d-43ec-9705-31ac0fa419de service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:56.520548 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:56.521444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:56.521723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:56.521992 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] PATH_INFO: `/v3/projects/4fe102d227ea4f60bdf22c20cd028341` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:56.523205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe102d227ea4f60bdf22c20cd028341)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:56.525159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:56.631193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe102d227ea4f60bdf22c20cd028341', 'request_id': 'req-26a3f120-b741-4871-9654-b041acc0e4b6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:56.637572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe102d227ea4f60bdf22c20cd028341', 'request_id': 'req-26a3f120-b741-4871-9654-b041acc0e4b6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:56.643244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe102d227ea4f60bdf22c20cd028341', 'request_id': 'req-26a3f120-b741-4871-9654-b041acc0e4b6'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:56.722760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] The token cache is being invalidate because project 4fe102d227ea4f60bdf22c20cd028341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:55:56.722992 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:56.723584 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-26a3f120-b741-4871-9654-b041acc0e4b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:55:56.725172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1282/2581] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:55:56 2022] DELETE /identity/v3/projects/4fe102d227ea4f60bdf22c20cd028341 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:55:56.730323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c952db1-a362-4091-bd28-e48371a090ed None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:56.730615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c952db1-a362-4091-bd28-e48371a090ed None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:56.730895 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c952db1-a362-4091-bd28-e48371a090ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:56.750340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2c952db1-a362-4091-bd28-e48371a090ed None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:55:56.751572 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2c952db1-a362-4091-bd28-e48371a090ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:56.811512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1300/2582] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:55:56.819697 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2316d0-b9fb-4fa4-964c-25d2cae2ed52 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:56.819990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2316d0-b9fb-4fa4-964c-25d2cae2ed52 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:56.820270 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2316d0-b9fb-4fa4-964c-25d2cae2ed52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:56.839794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-dd2316d0-b9fb-4fa4-964c-25d2cae2ed52 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:55:56.841072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dd2316d0-b9fb-4fa4-964c-25d2cae2ed52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:56.849789 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1283/2583] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:55:56.854020 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c952db1-a362-4091-bd28-e48371a090ed None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:56.864277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c952db1-a362-4091-bd28-e48371a090ed None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:56.869001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404d7fb-2ad9-45c9-b93b-593392905a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:56.869722 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404d7fb-2ad9-45c9-b93b-593392905a05 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:56.870104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404d7fb-2ad9-45c9-b93b-593392905a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:56.870388 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404d7fb-2ad9-45c9-b93b-593392905a05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:56.871126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404d7fb-2ad9-45c9-b93b-593392905a05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:55:56.871432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404d7fb-2ad9-45c9-b93b-593392905a05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:56.873119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7404d7fb-2ad9-45c9-b93b-593392905a05 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:56.874713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:55:56.886120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1301/2584] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:55:56.890527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2316d0-b9fb-4fa4-964c-25d2cae2ed52 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:56.900010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2316d0-b9fb-4fa4-964c-25d2cae2ed52 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:56.904779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d2573-e276-432f-9dac-b1e7688be590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:56.905447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d2573-e276-432f-9dac-b1e7688be590 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:56.905728 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d2573-e276-432f-9dac-b1e7688be590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:56.906000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d2573-e276-432f-9dac-b1e7688be590 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:56.906685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d2573-e276-432f-9dac-b1e7688be590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:56.908631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d2573-e276-432f-9dac-b1e7688be590 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:56.918818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:55:56.993439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1284/2585] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:55:56 2022] POST /identity/v3/projects => generated 399 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:55:56.997524 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404d7fb-2ad9-45c9-b93b-593392905a05 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:57.008232 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404d7fb-2ad9-45c9-b93b-593392905a05 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:57.065903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb661f-f977-408c-8038-5bcd42051c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:57.066583 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb661f-f977-408c-8038-5bcd42051c53 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:57.066871 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb661f-f977-408c-8038-5bcd42051c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:57.067151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb661f-f977-408c-8038-5bcd42051c53 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:57.067857 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb661f-f977-408c-8038-5bcd42051c53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:57.069875 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb661f-f977-408c-8038-5bcd42051c53 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:57.181265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1302/2586] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:55:56 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:55:57.185559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d2573-e276-432f-9dac-b1e7688be590 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:57.196509 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d2573-e276-432f-9dac-b1e7688be590 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:57.201086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3a29d-c614-40de-993c-e471ac513589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:57.202016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a29d-c614-40de-993c-e471ac513589 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:57.202588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a29d-c614-40de-993c-e471ac513589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:57.203052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3a29d-c614-40de-993c-e471ac513589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:57.204026 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3a29d-c614-40de-993c-e471ac513589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:57.206268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3a29d-c614-40de-993c-e471ac513589 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:57.216852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1285/2587] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:55:57.220988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb661f-f977-408c-8038-5bcd42051c53 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:57.231761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb661f-f977-408c-8038-5bcd42051c53 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:57.236689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c25b38-fd1b-4069-8bbc-c730fa4ca941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:57.237425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c25b38-fd1b-4069-8bbc-c730fa4ca941 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:57.237716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c25b38-fd1b-4069-8bbc-c730fa4ca941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:57.237993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c25b38-fd1b-4069-8bbc-c730fa4ca941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:57.238721 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c25b38-fd1b-4069-8bbc-c730fa4ca941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:57.240329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c25b38-fd1b-4069-8bbc-c730fa4ca941 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:57.250768 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1303/2588] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:55:57.255576 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3a29d-c614-40de-993c-e471ac513589 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:57.265812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3a29d-c614-40de-993c-e471ac513589 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:57.271557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663a025-5765-4d2c-82ea-377a24279f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:57.272442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663a025-5765-4d2c-82ea-377a24279f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:57.272729 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663a025-5765-4d2c-82ea-377a24279f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:57.273047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663a025-5765-4d2c-82ea-377a24279f96 admin admin] PATH_INFO: `/v3/projects/a92ae56e501f40f796b397b56531eed8/users/8d5ad1e266274bde9a0deb9a95aabf67/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:57.288276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663a025-5765-4d2c-82ea-377a24279f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a92ae56e501f40f796b397b56531eed8, user_id=8d5ad1e266274bde9a0deb9a95aabf67, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:57.291033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663a025-5765-4d2c-82ea-377a24279f96 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:57.373630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1286/2589] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:55:57 2022] PUT /identity/v3/projects/a92ae56e501f40f796b397b56531eed8/users/8d5ad1e266274bde9a0deb9a95aabf67/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:55:57.384122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d577d-4aab-4bba-8f17-289c77c2f749 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:57.384416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d577d-4aab-4bba-8f17-289c77c2f749 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:57.384701 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d577d-4aab-4bba-8f17-289c77c2f749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:57.413713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-551d577d-4aab-4bba-8f17-289c77c2f749 None None] MFA Rules not processed for user `8d5ad1e266274bde9a0deb9a95aabf67`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:55:57.414951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-551d577d-4aab-4bba-8f17-289c77c2f749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:57.485603 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1304/2590] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:55:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:55:57.497866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663a025-5765-4d2c-82ea-377a24279f96 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:57.499641 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e663a025-5765-4d2c-82ea-377a24279f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:57.557987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663a025-5765-4d2c-82ea-377a24279f96 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:57.561654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0155d-e09b-4211-85cd-d0ccbc622873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:57.562301 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0155d-e09b-4211-85cd-d0ccbc622873 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:57.567041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0155d-e09b-4211-85cd-d0ccbc622873 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:57.567041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0155d-e09b-4211-85cd-d0ccbc622873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:57.571551 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0155d-e09b-4211-85cd-d0ccbc622873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:57.573082 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0155d-e09b-4211-85cd-d0ccbc622873 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:57.583054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1287/2591] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:57 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:55:58.111944 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d577d-4aab-4bba-8f17-289c77c2f749 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:55:58.113487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-551d577d-4aab-4bba-8f17-289c77c2f749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:58.174183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d577d-4aab-4bba-8f17-289c77c2f749 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:55:58.177832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ccba4-aaf3-49c2-80b6-f8b0784467d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:55:58.178489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ccba4-aaf3-49c2-80b6-f8b0784467d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:55:58.178777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ccba4-aaf3-49c2-80b6-f8b0784467d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:55:58.179057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ccba4-aaf3-49c2-80b6-f8b0784467d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:55:58.180859 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-119ccba4-aaf3-49c2-80b6-f8b0784467d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:55:58.191046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ccba4-aaf3-49c2-80b6-f8b0784467d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:55:58.192510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ccba4-aaf3-49c2-80b6-f8b0784467d4 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:55:58.200457 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1305/2592] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:55:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:06.704869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0155d-e09b-4211-85cd-d0ccbc622873 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:06.767608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0155d-e09b-4211-85cd-d0ccbc622873 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:06.772811 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:06.773701 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:06.773989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:06.774268 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] PATH_INFO: `/v3/users/8d5ad1e266274bde9a0deb9a95aabf67` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:06.775471 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d5ad1e266274bde9a0deb9a95aabf67)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:06.777441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:06.939437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d5ad1e266274bde9a0deb9a95aabf67'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:06.945661 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d5ad1e266274bde9a0deb9a95aabf67'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:06.950864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:06.959316 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d5ad1e266274bde9a0deb9a95aabf67'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:07.019199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d5ad1e266274bde9a0deb9a95aabf67'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:07.019829 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d5ad1e266274bde9a0deb9a95aabf67'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:07.027054 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1288/2593] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:56:06 2022] DELETE /identity/v3/users/8d5ad1e266274bde9a0deb9a95aabf67 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:07.031624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fe532-8d55-4e7c-9201-aa7e79eea24a None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:07.032103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fe532-8d55-4e7c-9201-aa7e79eea24a None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:07.032207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fe532-8d55-4e7c-9201-aa7e79eea24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:07.040398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:07.051804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-cc1fe532-8d55-4e7c-9201-aa7e79eea24a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:07.053163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cc1fe532-8d55-4e7c-9201-aa7e79eea24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:07.114946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1306/2594] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:07.125128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:07.126621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:07.188344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99593175-045a-4bc5-84ca-9212603bdde6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:07.192769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a97314-8f3e-4b6c-baac-e3a7a03aa4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:07.193398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a97314-8f3e-4b6c-baac-e3a7a03aa4ac service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:07.193686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a97314-8f3e-4b6c-baac-e3a7a03aa4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:07.193965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a97314-8f3e-4b6c-baac-e3a7a03aa4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:07.203970 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a97314-8f3e-4b6c-baac-e3a7a03aa4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:07.206097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a97314-8f3e-4b6c-baac-e3a7a03aa4ac service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:07.216145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1289/2595] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:07.682593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fe532-8d55-4e7c-9201-aa7e79eea24a None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:07.687293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-cc1fe532-8d55-4e7c-9201-aa7e79eea24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:07.703133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fe532-8d55-4e7c-9201-aa7e79eea24a None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:07.710819 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:07.714288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:07.714288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:07.714288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] PATH_INFO: `/v3/projects/a92ae56e501f40f796b397b56531eed8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:07.715252 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a92ae56e501f40f796b397b56531eed8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:07.718406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:07.851022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a92ae56e501f40f796b397b56531eed8', 'request_id': 'req-27c37e68-544c-4386-877a-aeb041a21be0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:07.933517 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a92ae56e501f40f796b397b56531eed8', 'request_id': 'req-27c37e68-544c-4386-877a-aeb041a21be0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:07.945058 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a92ae56e501f40f796b397b56531eed8', 'request_id': 'req-27c37e68-544c-4386-877a-aeb041a21be0'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:07.960348 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] The token cache is being invalidate because project a92ae56e501f40f796b397b56531eed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:56:07.960771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:07.961330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:07.962997 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1307/2596] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:56:07 2022] DELETE /identity/v3/projects/a92ae56e501f40f796b397b56531eed8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:10.446172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a97314-8f3e-4b6c-baac-e3a7a03aa4ac service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:10.447590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-86a97314-8f3e-4b6c-baac-e3a7a03aa4ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:10.459097 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a97314-8f3e-4b6c-baac-e3a7a03aa4ac service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:10.513551 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e44acf-0fe0-4b70-be04-01891660c4de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:10.514274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e44acf-0fe0-4b70-be04-01891660c4de service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:10.514572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e44acf-0fe0-4b70-be04-01891660c4de service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:10.514849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e44acf-0fe0-4b70-be04-01891660c4de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:10.516678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f1e44acf-0fe0-4b70-be04-01891660c4de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:10.581840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e44acf-0fe0-4b70-be04-01891660c4de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:10.586740 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e44acf-0fe0-4b70-be04-01891660c4de service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:10.598180 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1290/2597] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:14.456347 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:14.457787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:14.471878 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c37e68-544c-4386-877a-aeb041a21be0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:14.476684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb59cc2-fdf7-45b0-83bc-0d6df4e1f527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:14.480628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb59cc2-fdf7-45b0-83bc-0d6df4e1f527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:14.480628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb59cc2-fdf7-45b0-83bc-0d6df4e1f527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:14.480628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb59cc2-fdf7-45b0-83bc-0d6df4e1f527 admin admin] PATH_INFO: `/v3/users/f2d1f86034b94e289bb1fc754f02f4ac` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:14.480628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb59cc2-fdf7-45b0-83bc-0d6df4e1f527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2d1f86034b94e289bb1fc754f02f4ac)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:14.481275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb59cc2-fdf7-45b0-83bc-0d6df4e1f527 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:14.685172 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9cb59cc2-fdf7-45b0-83bc-0d6df4e1f527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2d1f86034b94e289bb1fc754f02f4ac'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:14.697931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9cb59cc2-fdf7-45b0-83bc-0d6df4e1f527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2d1f86034b94e289bb1fc754f02f4ac'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:14.705855 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9cb59cc2-fdf7-45b0-83bc-0d6df4e1f527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2d1f86034b94e289bb1fc754f02f4ac'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:14.750830 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9cb59cc2-fdf7-45b0-83bc-0d6df4e1f527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2d1f86034b94e289bb1fc754f02f4ac'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:14.751485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9cb59cc2-fdf7-45b0-83bc-0d6df4e1f527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2d1f86034b94e289bb1fc754f02f4ac'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:14.754468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:14.758921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1308/2598] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:56:14 2022] DELETE /identity/v3/users/f2d1f86034b94e289bb1fc754f02f4ac => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:56:14.974677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e44acf-0fe0-4b70-be04-01891660c4de service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:14.985545 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e44acf-0fe0-4b70-be04-01891660c4de service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:15.018583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:15.046851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:15.047717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:15.047999 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:15.048272 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] PATH_INFO: `/v3/projects/6299ea3e56e84d3cbd2e8f74ef62510e` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:15.049512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6299ea3e56e84d3cbd2e8f74ef62510e)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:15.051438 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:15.150808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6299ea3e56e84d3cbd2e8f74ef62510e', 'request_id': 'req-589d0508-670f-4c9d-a2b7-99b3b6413806'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:15.157293 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6299ea3e56e84d3cbd2e8f74ef62510e', 'request_id': 'req-589d0508-670f-4c9d-a2b7-99b3b6413806'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:15.163614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6299ea3e56e84d3cbd2e8f74ef62510e', 'request_id': 'req-589d0508-670f-4c9d-a2b7-99b3b6413806'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:15.246820 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] The token cache is being invalidate because project 6299ea3e56e84d3cbd2e8f74ef62510e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:56:15.247151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:15.247804 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-589d0508-670f-4c9d-a2b7-99b3b6413806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:15.249420 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1291/2599] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:56:14 2022] DELETE /identity/v3/projects/6299ea3e56e84d3cbd2e8f74ef62510e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:15.254625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050816-9c9c-416b-b797-cae007f91b01 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:15.254859 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050816-9c9c-416b-b797-cae007f91b01 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:15.255141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050816-9c9c-416b-b797-cae007f91b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:15.273844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-89050816-9c9c-416b-b797-cae007f91b01 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:15.275110 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-89050816-9c9c-416b-b797-cae007f91b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:15.335496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1309/2600] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:56:15.344115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb5b44-fd7d-4719-9b0f-97a5f41c050d None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:15.344408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb5b44-fd7d-4719-9b0f-97a5f41c050d None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:15.344687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb5b44-fd7d-4719-9b0f-97a5f41c050d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:15.365383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-6bbb5b44-fd7d-4719-9b0f-97a5f41c050d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:15.366597 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6bbb5b44-fd7d-4719-9b0f-97a5f41c050d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:15.374596 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1292/2601] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:15.379013 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89050816-9c9c-416b-b797-cae007f91b01 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:15.389144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89050816-9c9c-416b-b797-cae007f91b01 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:15.393677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87dfdd-0d2a-4f99-a774-438ef14931fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:15.394323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87dfdd-0d2a-4f99-a774-438ef14931fa admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:15.394603 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87dfdd-0d2a-4f99-a774-438ef14931fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:15.394876 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87dfdd-0d2a-4f99-a774-438ef14931fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:15.395592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87dfdd-0d2a-4f99-a774-438ef14931fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:56:15.395891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87dfdd-0d2a-4f99-a774-438ef14931fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:15.397511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87dfdd-0d2a-4f99-a774-438ef14931fa admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:15.409996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1310/2602] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 16:56:15.414395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb5b44-fd7d-4719-9b0f-97a5f41c050d None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:15.424432 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb5b44-fd7d-4719-9b0f-97a5f41c050d None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:15.429267 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7e0e0-77a5-4ed9-9a9c-dfd5ded4b176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:15.429928 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7e0e0-77a5-4ed9-9a9c-dfd5ded4b176 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:15.430227 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7e0e0-77a5-4ed9-9a9c-dfd5ded4b176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:15.430491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7e0e0-77a5-4ed9-9a9c-dfd5ded4b176 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:15.431199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7e0e0-77a5-4ed9-9a9c-dfd5ded4b176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:15.433126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7e0e0-77a5-4ed9-9a9c-dfd5ded4b176 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:15.532912 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1293/2603] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:56:15 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:56:15.534565 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87dfdd-0d2a-4f99-a774-438ef14931fa admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:15.546157 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87dfdd-0d2a-4f99-a774-438ef14931fa admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:15.608091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5340425-bc5e-4dac-9f4a-b43c7bcec156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:15.608769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5340425-bc5e-4dac-9f4a-b43c7bcec156 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:15.609076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5340425-bc5e-4dac-9f4a-b43c7bcec156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:15.609350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5340425-bc5e-4dac-9f4a-b43c7bcec156 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:15.610038 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5340425-bc5e-4dac-9f4a-b43c7bcec156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:15.612052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5340425-bc5e-4dac-9f4a-b43c7bcec156 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:15.687589 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1311/2604] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:56:15 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:56:15.692302 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7e0e0-77a5-4ed9-9a9c-dfd5ded4b176 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:15.703315 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7e0e0-77a5-4ed9-9a9c-dfd5ded4b176 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:15.708018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57731c8-c833-46e8-9a4a-31393f074a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:15.708686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57731c8-c833-46e8-9a4a-31393f074a7a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:15.709025 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57731c8-c833-46e8-9a4a-31393f074a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:15.709329 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57731c8-c833-46e8-9a4a-31393f074a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:15.710042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57731c8-c833-46e8-9a4a-31393f074a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:15.711659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57731c8-c833-46e8-9a4a-31393f074a7a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:15.722669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1294/2605] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:56:15.726761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5340425-bc5e-4dac-9f4a-b43c7bcec156 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:15.737783 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5340425-bc5e-4dac-9f4a-b43c7bcec156 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:15.742950 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82cf9e-2de4-4b5e-821e-0d2300500ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:15.743866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82cf9e-2de4-4b5e-821e-0d2300500ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:15.743866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82cf9e-2de4-4b5e-821e-0d2300500ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:15.744160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82cf9e-2de4-4b5e-821e-0d2300500ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:15.744927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82cf9e-2de4-4b5e-821e-0d2300500ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:15.746559 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82cf9e-2de4-4b5e-821e-0d2300500ccc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:15.758033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1312/2606] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:56:15.762366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57731c8-c833-46e8-9a4a-31393f074a7a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:15.772461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57731c8-c833-46e8-9a4a-31393f074a7a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:15.777061 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ce404-86d4-4958-a731-8184db0693a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:15.777939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ce404-86d4-4958-a731-8184db0693a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:15.778216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ce404-86d4-4958-a731-8184db0693a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:15.778497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ce404-86d4-4958-a731-8184db0693a0 admin admin] PATH_INFO: `/v3/projects/034f143ba31148c0939a06296ec18a96/users/24b3c774c1b04537a970c1e7dfdd6ac3/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:15.793079 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ce404-86d4-4958-a731-8184db0693a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=034f143ba31148c0939a06296ec18a96, user_id=24b3c774c1b04537a970c1e7dfdd6ac3, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:15.795624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ce404-86d4-4958-a731-8184db0693a0 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:15.877540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1295/2607] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:56:15 2022] PUT /identity/v3/projects/034f143ba31148c0939a06296ec18a96/users/24b3c774c1b04537a970c1e7dfdd6ac3/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:15.882586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350caf56-022d-4455-ba0d-f9d71575e4cd None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:15.882885 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350caf56-022d-4455-ba0d-f9d71575e4cd None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:15.883172 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350caf56-022d-4455-ba0d-f9d71575e4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:15.903151 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-350caf56-022d-4455-ba0d-f9d71575e4cd None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:15.904416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-350caf56-022d-4455-ba0d-f9d71575e4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:15.979246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1313/2608] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:15.992445 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ce404-86d4-4958-a731-8184db0693a0 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:15.994223 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-886ce404-86d4-4958-a731-8184db0693a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:16.051334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ce404-86d4-4958-a731-8184db0693a0 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:16.055393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829fa90-b30e-4459-9cd9-8c0e48daaa01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:16.056033 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829fa90-b30e-4459-9cd9-8c0e48daaa01 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:16.056343 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829fa90-b30e-4459-9cd9-8c0e48daaa01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:16.056616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829fa90-b30e-4459-9cd9-8c0e48daaa01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:16.066310 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829fa90-b30e-4459-9cd9-8c0e48daaa01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:16.067748 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829fa90-b30e-4459-9cd9-8c0e48daaa01 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:16.072667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:16.077920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1296/2609] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:21.871914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e12fc1-85db-4489-882c-1c2335d31b19 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:21.871914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e12fc1-85db-4489-882c-1c2335d31b19 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:21.872839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e12fc1-85db-4489-882c-1c2335d31b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:21.886945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:21.919380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-84e12fc1-85db-4489-882c-1c2335d31b19 None None] MFA Rules not processed for user `24b3c774c1b04537a970c1e7dfdd6ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:21.920854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-84e12fc1-85db-4489-882c-1c2335d31b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:22.018181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1314/2610] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:56:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:22.032256 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829fa90-b30e-4459-9cd9-8c0e48daaa01 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:22.035305 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0829fa90-b30e-4459-9cd9-8c0e48daaa01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:22.178421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829fa90-b30e-4459-9cd9-8c0e48daaa01 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:22.190716 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5892b3-947d-4fec-b5ba-d298a3979db8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:22.191664 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5892b3-947d-4fec-b5ba-d298a3979db8 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:22.192112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5892b3-947d-4fec-b5ba-d298a3979db8 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:22.192538 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5892b3-947d-4fec-b5ba-d298a3979db8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:22.220178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5892b3-947d-4fec-b5ba-d298a3979db8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:22.222439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5892b3-947d-4fec-b5ba-d298a3979db8 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:22.233543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1297/2611] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:25.039879 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3996e-f570-42c4-99f9-4c7baeadb0e0 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:25.041488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3996e-f570-42c4-99f9-4c7baeadb0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:25.041775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3996e-f570-42c4-99f9-4c7baeadb0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:25.064569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-0fe3996e-f570-42c4-99f9-4c7baeadb0e0 None None] MFA Rules not processed for user `24b3c774c1b04537a970c1e7dfdd6ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:25.065844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0fe3996e-f570-42c4-99f9-4c7baeadb0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:25.074191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1315/2612] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:56:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:25.448839 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5892b3-947d-4fec-b5ba-d298a3979db8 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:25.463679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5892b3-947d-4fec-b5ba-d298a3979db8 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:25.468016 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00993a9f-2bed-4195-9c18-7e9f3c466e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:25.469533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00993a9f-2bed-4195-9c18-7e9f3c466e39 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:25.469533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00993a9f-2bed-4195-9c18-7e9f3c466e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:25.469533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00993a9f-2bed-4195-9c18-7e9f3c466e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:25.471242 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-00993a9f-2bed-4195-9c18-7e9f3c466e39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:25.482586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00993a9f-2bed-4195-9c18-7e9f3c466e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:25.484484 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00993a9f-2bed-4195-9c18-7e9f3c466e39 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:25.496465 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1298/2613] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:25.962575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3996e-f570-42c4-99f9-4c7baeadb0e0 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:25.964843 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0fe3996e-f570-42c4-99f9-4c7baeadb0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:26.062742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3996e-f570-42c4-99f9-4c7baeadb0e0 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:26.069234 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8c44c-4ba6-43fe-9e1c-577c797e0c66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:26.069887 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c44c-4ba6-43fe-9e1c-577c797e0c66 service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:26.070177 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c44c-4ba6-43fe-9e1c-577c797e0c66 service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:26.070452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c44c-4ba6-43fe-9e1c-577c797e0c66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:26.072282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-beb8c44c-4ba6-43fe-9e1c-577c797e0c66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:26.086667 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8c44c-4ba6-43fe-9e1c-577c797e0c66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:26.088294 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8c44c-4ba6-43fe-9e1c-577c797e0c66 service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:26.100259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1316/2614] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:31.403259 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911bac3-0631-4565-9175-981e539d658b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:31.406135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911bac3-0631-4565-9175-981e539d658b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:31.406135 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911bac3-0631-4565-9175-981e539d658b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:31.424625 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-4911bac3-0631-4565-9175-981e539d658b None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:31.425911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4911bac3-0631-4565-9175-981e539d658b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:31.516619 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1299/2615] 10.222.0.88 () {58 vars in 967 bytes} [Mon Jul 18 16:56:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:31.567868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8c44c-4ba6-43fe-9e1c-577c797e0c66 service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:31.569358 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-beb8c44c-4ba6-43fe-9e1c-577c797e0c66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:31.645148 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8c44c-4ba6-43fe-9e1c-577c797e0c66 service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:31.650711 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7935c-af9d-4434-abd5-5180b889a8da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:31.651410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7935c-af9d-4434-abd5-5180b889a8da service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:31.651703 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7935c-af9d-4434-abd5-5180b889a8da service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:31.651989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7935c-af9d-4434-abd5-5180b889a8da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:31.661983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7935c-af9d-4434-abd5-5180b889a8da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:31.663773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7935c-af9d-4434-abd5-5180b889a8da service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:31.672495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1317/2616] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:56:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:56:37.983923 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911bac3-0631-4565-9175-981e539d658b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:37.986027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4911bac3-0631-4565-9175-981e539d658b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:38.004759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911bac3-0631-4565-9175-981e539d658b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:38.011644 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bfade-d9ef-4585-959c-8b560789ea29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:38.012478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bfade-d9ef-4585-959c-8b560789ea29 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:38.012833 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bfade-d9ef-4585-959c-8b560789ea29 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:38.013404 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bfade-d9ef-4585-959c-8b560789ea29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:38.015650 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6c1bfade-d9ef-4585-959c-8b560789ea29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:38.033330 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1bfade-d9ef-4585-959c-8b560789ea29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:38.035294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1bfade-d9ef-4585-959c-8b560789ea29 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:38.050014 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1300/2617] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:38.631636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7935c-af9d-4434-abd5-5180b889a8da service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:38.645895 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7935c-af9d-4434-abd5-5180b889a8da service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:38.651887 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5b9a5-5f2f-4066-84c4-0f6748e6dc31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:38.652654 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5b9a5-5f2f-4066-84c4-0f6748e6dc31 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:38.653195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5b9a5-5f2f-4066-84c4-0f6748e6dc31 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:38.653557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5b9a5-5f2f-4066-84c4-0f6748e6dc31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:38.655406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-56a5b9a5-5f2f-4066-84c4-0f6748e6dc31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:38.667735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=100 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:38.670663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5b9a5-5f2f-4066-84c4-0f6748e6dc31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:38.672498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5b9a5-5f2f-4066-84c4-0f6748e6dc31 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:38.684945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1318/2618] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:42.862243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bfade-d9ef-4585-959c-8b560789ea29 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:42.864350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6c1bfade-d9ef-4585-959c-8b560789ea29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:42.880633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bfade-d9ef-4585-959c-8b560789ea29 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:42.886153 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:42.887079 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:42.887360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:42.887640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] PATH_INFO: `/v3/users/ab4d3667826148d390eda90bb8baf081` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:42.888961 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4d3667826148d390eda90bb8baf081)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:42.890969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:43.124412 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4d3667826148d390eda90bb8baf081'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:43.133982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4d3667826148d390eda90bb8baf081'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:43.137138 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:43.152702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4d3667826148d390eda90bb8baf081'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:43.222158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4d3667826148d390eda90bb8baf081'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:43.227290 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4d3667826148d390eda90bb8baf081'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:43.240422 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1301/2619] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:56:42 2022] DELETE /identity/v3/users/ab4d3667826148d390eda90bb8baf081 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:43.883314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5b9a5-5f2f-4066-84c4-0f6748e6dc31 service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:43.916276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5b9a5-5f2f-4066-84c4-0f6748e6dc31 service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:44.007816 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:44.008798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:44.009123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:44.009422 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] PATH_INFO: `/v3/projects/a0aa59ff7f5a460daa866b1f3e6dae1a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:44.010613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0aa59ff7f5a460daa866b1f3e6dae1a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:44.013173 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:44.133790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0aa59ff7f5a460daa866b1f3e6dae1a', 'request_id': 'req-82ee302f-94f8-420a-98ef-ad2adb8ec723'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:44.229810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0aa59ff7f5a460daa866b1f3e6dae1a', 'request_id': 'req-82ee302f-94f8-420a-98ef-ad2adb8ec723'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:44.238003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0aa59ff7f5a460daa866b1f3e6dae1a', 'request_id': 'req-82ee302f-94f8-420a-98ef-ad2adb8ec723'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:44.250531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] The token cache is being invalidate because project a0aa59ff7f5a460daa866b1f3e6dae1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:56:44.251399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:44.252082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:44.253680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1319/2620] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:56:43 2022] DELETE /identity/v3/projects/a0aa59ff7f5a460daa866b1f3e6dae1a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:44.258091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:44.259502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:44.273678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da9c2e-df7f-4098-8d30-faa440939694 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:44.352377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:44.356073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:44.356073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:44.356073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] PATH_INFO: `/v3/users/dc1e3642b7a24808ba8e4160a0f66ba7` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:44.356073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc1e3642b7a24808ba8e4160a0f66ba7)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:44.358238 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:44.635700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc1e3642b7a24808ba8e4160a0f66ba7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:44.643810 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc1e3642b7a24808ba8e4160a0f66ba7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:44.658282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc1e3642b7a24808ba8e4160a0f66ba7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:44.732427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc1e3642b7a24808ba8e4160a0f66ba7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:44.733041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc1e3642b7a24808ba8e4160a0f66ba7'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:44.745064 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1302/2621] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:56:44 2022] DELETE /identity/v3/users/dc1e3642b7a24808ba8e4160a0f66ba7 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:45.088807 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:45.091573 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:45.106325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee302f-94f8-420a-98ef-ad2adb8ec723 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:45.170372 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:45.171166 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:45.171442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:45.171710 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] PATH_INFO: `/v3/projects/b3c553fb947148fcbfcdab882d77ae3d` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:45.173608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c553fb947148fcbfcdab882d77ae3d)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:45.175529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:45.335374 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:45.350649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c553fb947148fcbfcdab882d77ae3d', 'request_id': 'req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:45.440840 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c553fb947148fcbfcdab882d77ae3d', 'request_id': 'req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:45.447102 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c553fb947148fcbfcdab882d77ae3d', 'request_id': 'req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:45.462560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] The token cache is being invalidate because project b3c553fb947148fcbfcdab882d77ae3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:56:45.462850 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:45.463504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:45.465171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1320/2622] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:56:45 2022] DELETE /identity/v3/projects/b3c553fb947148fcbfcdab882d77ae3d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:46.007247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:46.008725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:46.111181 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5911c3-4730-471a-a556-ec55534dc524 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:46.128039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:46.179104 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f477446-bf8e-47ad-baae-40026f700e62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:46.179764 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f477446-bf8e-47ad-baae-40026f700e62 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:46.180047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f477446-bf8e-47ad-baae-40026f700e62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:46.180319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f477446-bf8e-47ad-baae-40026f700e62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:46.182219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5f477446-bf8e-47ad-baae-40026f700e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:46.268796 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f477446-bf8e-47ad-baae-40026f700e62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:46.270579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f477446-bf8e-47ad-baae-40026f700e62 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:46.283440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1303/2623] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:51.265199 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:51.267216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:51.280030 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c87670-5ad7-4ab4-b3e6-6e8060c62ef5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:51.283589 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584b326-166c-455f-aec6-ab5f5529da22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:51.284180 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584b326-166c-455f-aec6-ab5f5529da22 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:51.284735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584b326-166c-455f-aec6-ab5f5529da22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:51.284735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584b326-166c-455f-aec6-ab5f5529da22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:51.286577 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3584b326-166c-455f-aec6-ab5f5529da22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:51.299707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584b326-166c-455f-aec6-ab5f5529da22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:51.301191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584b326-166c-455f-aec6-ab5f5529da22 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:51.311551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1321/2624] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:54.494942 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f477446-bf8e-47ad-baae-40026f700e62 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:54.496748 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-5f477446-bf8e-47ad-baae-40026f700e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:54.574062 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f477446-bf8e-47ad-baae-40026f700e62 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:54.579446 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c08d8-12ef-443c-b3ca-0f740c2443b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:54.581410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c08d8-12ef-443c-b3ca-0f740c2443b8 service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:54.581410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c08d8-12ef-443c-b3ca-0f740c2443b8 service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:54.581410 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c08d8-12ef-443c-b3ca-0f740c2443b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:54.584081 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ab3c08d8-12ef-443c-b3ca-0f740c2443b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:54.600514 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c08d8-12ef-443c-b3ca-0f740c2443b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:54.603200 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c08d8-12ef-443c-b3ca-0f740c2443b8 service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:54.613624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1304/2625] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:56:54.695592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584b326-166c-455f-aec6-ab5f5529da22 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:54.697501 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3584b326-166c-455f-aec6-ab5f5529da22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:54.711032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584b326-166c-455f-aec6-ab5f5529da22 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:54.716154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:54.717448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:54.717448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:54.717662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] PATH_INFO: `/v3/users/8016e561397f4b4faca9e03343d51355` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:54.718902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8016e561397f4b4faca9e03343d51355)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:54.720990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:54.878607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8016e561397f4b4faca9e03343d51355'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:54.892431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8016e561397f4b4faca9e03343d51355'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:54.899942 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8016e561397f4b4faca9e03343d51355'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:54.989178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8016e561397f4b4faca9e03343d51355'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:54.989885 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8016e561397f4b4faca9e03343d51355'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:54.996490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1322/2626] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:56:54 2022] DELETE /identity/v3/users/8016e561397f4b4faca9e03343d51355 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:55.299675 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c08d8-12ef-443c-b3ca-0f740c2443b8 service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:55.312186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c08d8-12ef-443c-b3ca-0f740c2443b8 service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:55.368812 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=100 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:55.379575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:55.380532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:55.380947 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:55.381152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] PATH_INFO: `/v3/projects/e3e2c4fb7f354179a90bfa7d963aa50c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:55.382442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e2c4fb7f354179a90bfa7d963aa50c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:55.384377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:55.522769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e2c4fb7f354179a90bfa7d963aa50c', 'request_id': 'req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:55.529491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e2c4fb7f354179a90bfa7d963aa50c', 'request_id': 'req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:55.535691 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e2c4fb7f354179a90bfa7d963aa50c', 'request_id': 'req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:55.625989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] The token cache is being invalidate because project e3e2c4fb7f354179a90bfa7d963aa50c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:56:55.625989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:55.625989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:55.625989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1305/2627] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:56:55 2022] DELETE /identity/v3/projects/e3e2c4fb7f354179a90bfa7d963aa50c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:55.627462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:55.629124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:55.654745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b58a2-573a-4dbb-aeac-717153423110 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:55.726584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:55.727468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:55.727755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:55.728023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] PATH_INFO: `/v3/users/5dc028113e864ab7bb24693cad914011` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:55.729257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc028113e864ab7bb24693cad914011)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:55.731301 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:55.747813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:55.784695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:55.786208 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:55.800236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c99972f-95e7-4ba1-8b9a-d8d113c75a5a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:55.805073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:55.805950 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:55.806269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:55.806549 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] PATH_INFO: `/v3/users/6890f2e62cfd46999791d4a10a81f95c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:55.807766 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6890f2e62cfd46999791d4a10a81f95c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:55.809751 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:55.957674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc028113e864ab7bb24693cad914011'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:55.969443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc028113e864ab7bb24693cad914011'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:55.976504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc028113e864ab7bb24693cad914011'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.036832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6890f2e62cfd46999791d4a10a81f95c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.044108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6890f2e62cfd46999791d4a10a81f95c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.052363 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc028113e864ab7bb24693cad914011'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.053273 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc028113e864ab7bb24693cad914011'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.057031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6890f2e62cfd46999791d4a10a81f95c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.060079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1323/2628] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:56:55 2022] DELETE /identity/v3/users/5dc028113e864ab7bb24693cad914011 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:56.155090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6890f2e62cfd46999791d4a10a81f95c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.155765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6890f2e62cfd46999791d4a10a81f95c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.162917 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1306/2629] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 16:56:55 2022] DELETE /identity/v3/users/6890f2e62cfd46999791d4a10a81f95c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 16:56:56.317496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:56.322222 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:56.405854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dd0db-ccf7-4271-8920-ac590f48a74d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:56.414084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:56.416281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:56.473665 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:56.474627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:56.475002 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:56.475407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] PATH_INFO: `/v3/projects/0c899dbcca564df69e3a2b58a3fe50c8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:56.476685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c899dbcca564df69e3a2b58a3fe50c8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:56.478683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:56.482376 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:56.494677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a10819-8983-4e5f-a25a-d60faebbea1d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:56.499494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:56.500896 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:56.501188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:56.501460 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] PATH_INFO: `/v3/projects/e8bca7eae3a348069131e97b048412b5` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:56.502666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8bca7eae3a348069131e97b048412b5)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:56.504582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:56.592534 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:56.596376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c899dbcca564df69e3a2b58a3fe50c8', 'request_id': 'req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.622640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8bca7eae3a348069131e97b048412b5', 'request_id': 'req-cea39b23-1788-487a-9222-f930883a0955'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.629581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8bca7eae3a348069131e97b048412b5', 'request_id': 'req-cea39b23-1788-487a-9222-f930883a0955'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.635111 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8bca7eae3a348069131e97b048412b5', 'request_id': 'req-cea39b23-1788-487a-9222-f930883a0955'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.692197 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c899dbcca564df69e3a2b58a3fe50c8', 'request_id': 'req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.701663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c899dbcca564df69e3a2b58a3fe50c8', 'request_id': 'req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.711791 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] The token cache is being invalidate because project 0c899dbcca564df69e3a2b58a3fe50c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:56:56.712075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.712743 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8bd47b7e-8cb2-40ea-92da-0c88a40a68ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.714317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1324/2630] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:56:56 2022] DELETE /identity/v3/projects/0c899dbcca564df69e3a2b58a3fe50c8 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:56.766622 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] The token cache is being invalidate because project e8bca7eae3a348069131e97b048412b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 16:56:56.767258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.768121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-cea39b23-1788-487a-9222-f930883a0955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 16:56:56.769816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1307/2631] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 16:56:56 2022] DELETE /identity/v3/projects/e8bca7eae3a348069131e97b048412b5 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:56.776683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9844c5d-ec47-4fc8-962c-4a0d052de946 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:56.780045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9844c5d-ec47-4fc8-962c-4a0d052de946 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:56.780045 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9844c5d-ec47-4fc8-962c-4a0d052de946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:56.798379 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e9844c5d-ec47-4fc8-962c-4a0d052de946 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:56.799852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e9844c5d-ec47-4fc8-962c-4a0d052de946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:56.869448 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1325/2632] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:56.879194 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518308f6-c8a8-4d9c-a5f1-9018fec1d713 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:56.879500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518308f6-c8a8-4d9c-a5f1-9018fec1d713 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:56.879781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518308f6-c8a8-4d9c-a5f1-9018fec1d713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:56.900680 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-518308f6-c8a8-4d9c-a5f1-9018fec1d713 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:56.902058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-518308f6-c8a8-4d9c-a5f1-9018fec1d713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:56.911319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1308/2633] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:56.915606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9844c5d-ec47-4fc8-962c-4a0d052de946 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:56.926506 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9844c5d-ec47-4fc8-962c-4a0d052de946 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:56.931247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c199ff-9fdd-447b-bfbf-b47e63e66ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:56.931916 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c199ff-9fdd-447b-bfbf-b47e63e66ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:56.932204 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c199ff-9fdd-447b-bfbf-b47e63e66ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:56.932484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c199ff-9fdd-447b-bfbf-b47e63e66ce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:56.934297 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c199ff-9fdd-447b-bfbf-b47e63e66ce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 16:56:56.934297 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c199ff-9fdd-447b-bfbf-b47e63e66ce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:56.935740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c199ff-9fdd-447b-bfbf-b47e63e66ce0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:56.948774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1326/2634] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 16:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 16:56:56.953763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518308f6-c8a8-4d9c-a5f1-9018fec1d713 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:56.962953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518308f6-c8a8-4d9c-a5f1-9018fec1d713 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:56.967649 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf567b9-70b9-45a8-9ded-9d648d1d1088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:56.968294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf567b9-70b9-45a8-9ded-9d648d1d1088 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:56.968572 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf567b9-70b9-45a8-9ded-9d648d1d1088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:56.968898 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf567b9-70b9-45a8-9ded-9d648d1d1088 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:56.969600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf567b9-70b9-45a8-9ded-9d648d1d1088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:56.971488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf567b9-70b9-45a8-9ded-9d648d1d1088 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:57.031945 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1309/2635] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 16:56:56 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:56:57.034532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c199ff-9fdd-447b-bfbf-b47e63e66ce0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:57.045439 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c199ff-9fdd-447b-bfbf-b47e63e66ce0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:57.114120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81a094-0e19-41f6-8cbd-0adfb1418aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:57.114796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81a094-0e19-41f6-8cbd-0adfb1418aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:57.115075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81a094-0e19-41f6-8cbd-0adfb1418aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:57.115349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81a094-0e19-41f6-8cbd-0adfb1418aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:57.116056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81a094-0e19-41f6-8cbd-0adfb1418aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:57.118007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81a094-0e19-41f6-8cbd-0adfb1418aa8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:57.212073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1327/2636] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 16:56:57 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 16:56:57.216318 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf567b9-70b9-45a8-9ded-9d648d1d1088 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:57.227585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf567b9-70b9-45a8-9ded-9d648d1d1088 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:57.232307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a55a34-b06b-48ea-a420-a581e53d24a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:57.233002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a55a34-b06b-48ea-a420-a581e53d24a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:57.233288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a55a34-b06b-48ea-a420-a581e53d24a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:57.233564 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a55a34-b06b-48ea-a420-a581e53d24a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:57.234294 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a55a34-b06b-48ea-a420-a581e53d24a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:57.236506 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a55a34-b06b-48ea-a420-a581e53d24a9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:57.247247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1310/2637] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:56:57.251854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81a094-0e19-41f6-8cbd-0adfb1418aa8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:57.262375 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81a094-0e19-41f6-8cbd-0adfb1418aa8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:57.267804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b646e3-508e-4cbe-8d30-e276a100d7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:57.268463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b646e3-508e-4cbe-8d30-e276a100d7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:57.268776 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b646e3-508e-4cbe-8d30-e276a100d7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:57.269085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b646e3-508e-4cbe-8d30-e276a100d7d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:57.269820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b646e3-508e-4cbe-8d30-e276a100d7d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:57.271416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b646e3-508e-4cbe-8d30-e276a100d7d2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:57.281735 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1328/2638] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 16:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 16:56:57.285967 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a55a34-b06b-48ea-a420-a581e53d24a9 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:57.296263 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a55a34-b06b-48ea-a420-a581e53d24a9 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:57.301087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c824e-9f11-4ee7-8321-43339fcbaf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:57.301962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c824e-9f11-4ee7-8321-43339fcbaf1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:57.302243 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c824e-9f11-4ee7-8321-43339fcbaf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:57.302516 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c824e-9f11-4ee7-8321-43339fcbaf1e admin admin] PATH_INFO: `/v3/projects/25de8ee0667645a3864d846675d460e1/users/730d6426b4ee4cf38893fe2e60b97230/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:57.317424 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c824e-9f11-4ee7-8321-43339fcbaf1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25de8ee0667645a3864d846675d460e1, user_id=730d6426b4ee4cf38893fe2e60b97230, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:57.320084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c824e-9f11-4ee7-8321-43339fcbaf1e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:57.389707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1311/2639] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 16:56:57 2022] PUT /identity/v3/projects/25de8ee0667645a3864d846675d460e1/users/730d6426b4ee4cf38893fe2e60b97230/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 16:56:57.395642 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6691-ed35-48fe-8b39-38bdc0cebfaf None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:57.395935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6691-ed35-48fe-8b39-38bdc0cebfaf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:57.396212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f6691-ed35-48fe-8b39-38bdc0cebfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:57.414269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b93f6691-ed35-48fe-8b39-38bdc0cebfaf None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:56:57.415465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b93f6691-ed35-48fe-8b39-38bdc0cebfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:57.449142 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:56:57.477461 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1329/2640] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:56:57.488066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c824e-9f11-4ee7-8321-43339fcbaf1e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:56:57.489942 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b61c824e-9f11-4ee7-8321-43339fcbaf1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:56:57.549758 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c824e-9f11-4ee7-8321-43339fcbaf1e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:56:57.554479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da118a55-5878-4ac2-af56-612460846d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:56:57.554479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da118a55-5878-4ac2-af56-612460846d23 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:56:57.554479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da118a55-5878-4ac2-af56-612460846d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:56:57.554781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da118a55-5878-4ac2-af56-612460846d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:56:57.564867 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da118a55-5878-4ac2-af56-612460846d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:56:57.566343 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da118a55-5878-4ac2-af56-612460846d23 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:56:57.576384 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1312/2641] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:56:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:57:02.946490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd2312-3d34-45a9-bfa1-d585e17cad36 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:02.947075 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd2312-3d34-45a9-bfa1-d585e17cad36 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:02.947404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cd2312-3d34-45a9-bfa1-d585e17cad36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:02.991601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-76cd2312-3d34-45a9-bfa1-d585e17cad36 None None] MFA Rules not processed for user `730d6426b4ee4cf38893fe2e60b97230`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:57:02.993525 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-76cd2312-3d34-45a9-bfa1-d585e17cad36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:03.076350 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1330/2642] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:57:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:57:03.090820 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da118a55-5878-4ac2-af56-612460846d23 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:57:03.092377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-da118a55-5878-4ac2-af56-612460846d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:03.129786 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:57:03.191390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da118a55-5878-4ac2-af56-612460846d23 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:57:03.198313 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12cf09-cbb6-42c5-bccc-42aeedcd6e69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:57:03.198969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12cf09-cbb6-42c5-bccc-42aeedcd6e69 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:03.199521 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12cf09-cbb6-42c5-bccc-42aeedcd6e69 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:03.199799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12cf09-cbb6-42c5-bccc-42aeedcd6e69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:03.207958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12cf09-cbb6-42c5-bccc-42aeedcd6e69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:57:03.209602 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12cf09-cbb6-42c5-bccc-42aeedcd6e69 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:57:03.218472 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1313/2643] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:57:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:57:05.666297 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a1c0b-4815-4692-9e6a-d9ccd7c8b184 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:05.667430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a1c0b-4815-4692-9e6a-d9ccd7c8b184 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:05.667889 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a1c0b-4815-4692-9e6a-d9ccd7c8b184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:05.695352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b77a1c0b-4815-4692-9e6a-d9ccd7c8b184 None None] MFA Rules not processed for user `730d6426b4ee4cf38893fe2e60b97230`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:57:05.697467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b77a1c0b-4815-4692-9e6a-d9ccd7c8b184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:05.706221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1331/2644] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:57:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:57:06.058125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12cf09-cbb6-42c5-bccc-42aeedcd6e69 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:57:06.069248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12cf09-cbb6-42c5-bccc-42aeedcd6e69 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:57:06.073262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c8335-9448-4f24-a268-76ad4a3eccf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:57:06.076631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c8335-9448-4f24-a268-76ad4a3eccf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:06.076631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c8335-9448-4f24-a268-76ad4a3eccf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:06.076631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c8335-9448-4f24-a268-76ad4a3eccf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:06.076631 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-fc8c8335-9448-4f24-a268-76ad4a3eccf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:06.088370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c8335-9448-4f24-a268-76ad4a3eccf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:57:06.090024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c8335-9448-4f24-a268-76ad4a3eccf5 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:57:06.099932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1314/2645] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:57:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:57:15.682812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=45 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:57:16.096211 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=30 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:57:22.558113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c273987-89bc-4aff-adc0-f477889a8005 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:22.562223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c273987-89bc-4aff-adc0-f477889a8005 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:22.562223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c273987-89bc-4aff-adc0-f477889a8005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:22.576706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4c273987-89bc-4aff-adc0-f477889a8005 None None] MFA Rules not processed for user `cd69c5111dac492a9177128317f0962a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:57:22.578072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4c273987-89bc-4aff-adc0-f477889a8005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:22.635313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1332/2646] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:57:32.628181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=25 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:57:42.969446 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a10a55-d6ca-459e-bad1-05397f12ff50 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:42.969919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a10a55-d6ca-459e-bad1-05397f12ff50 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:42.970028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a10a55-d6ca-459e-bad1-05397f12ff50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:42.989841 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-37a10a55-d6ca-459e-bad1-05397f12ff50 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:57:42.991019 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-37a10a55-d6ca-459e-bad1-05397f12ff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:43.061005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1315/2647] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:57:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:57:43.210072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c273987-89bc-4aff-adc0-f477889a8005 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:57:43.211552 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4c273987-89bc-4aff-adc0-f477889a8005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:43.286591 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c273987-89bc-4aff-adc0-f477889a8005 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:57:43.291666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca405c-114a-4e06-82a9-23762437857b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:57:43.292324 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca405c-114a-4e06-82a9-23762437857b service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:43.292610 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca405c-114a-4e06-82a9-23762437857b service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:43.292926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca405c-114a-4e06-82a9-23762437857b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:43.301407 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca405c-114a-4e06-82a9-23762437857b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:57:43.303004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca405c-114a-4e06-82a9-23762437857b service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:57:43.316676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1333/2648] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:57:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:57:53.054388 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=25 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:57:53.312989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=32 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:57:59.773334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e62494-6275-4e42-898c-1e218b141292 None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:59.773869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e62494-6275-4e42-898c-1e218b141292 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:59.773869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e62494-6275-4e42-898c-1e218b141292 None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:59.775281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1316/2649] 10.222.0.88 () {54 vars in 921 bytes} [Mon Jul 18 16:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:57:59.780903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db573f-ca12-43e1-b86b-7cad12d58a26 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:59.781335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db573f-ca12-43e1-b86b-7cad12d58a26 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:59.781499 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db573f-ca12-43e1-b86b-7cad12d58a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:57:59.846558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-79db573f-ca12-43e1-b86b-7cad12d58a26 None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:57:59.847714 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-79db573f-ca12-43e1-b86b-7cad12d58a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:59.920190 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1334/2650] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Jul 18 16:57:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:57:59.932585 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e62494-6275-4e42-898c-1e218b141292 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:57:59.934983 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-52e62494-6275-4e42-898c-1e218b141292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:57:59.994399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e62494-6275-4e42-898c-1e218b141292 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:57:59.998120 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a8d66-75a9-43a3-8dfa-17927363b5dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54c7869983bf4534b3e730af0185eac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:57:59.998757 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8d66-75a9-43a3-8dfa-17927363b5dc service glance] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:57:59.999039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8d66-75a9-43a3-8dfa-17927363b5dc service glance] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:57:59.999312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8d66-75a9-43a3-8dfa-17927363b5dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:00.010141 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a8d66-75a9-43a3-8dfa-17927363b5dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:58:00.011580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a8d66-75a9-43a3-8dfa-17927363b5dc service glance] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:58:00.023056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1317/2651] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:57:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 16:58:00.237136 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ac36d-8035-4854-9883-7cd0a7215268 None None] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:00.237136 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ac36d-8035-4854-9883-7cd0a7215268 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:00.237523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ac36d-8035-4854-9883-7cd0a7215268 None None] PATH_INFO: `/` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:00.238843 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1335/2652] 10.222.0.88 () {54 vars in 921 bytes} [Mon Jul 18 16:58:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:58:00.244173 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238abc3-2883-44f3-8cd4-433837ffe86e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:00.244463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238abc3-2883-44f3-8cd4-433837ffe86e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:00.249742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238abc3-2883-44f3-8cd4-433837ffe86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:00.263684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c238abc3-2883-44f3-8cd4-433837ffe86e None None] MFA Rules not processed for user `de8484c3b114467ea59bebb2460ed546`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:58:00.264944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c238abc3-2883-44f3-8cd4-433837ffe86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:58:00.271905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1318/2653] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Jul 18 16:58:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 16:58:00.284729 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ac36d-8035-4854-9883-7cd0a7215268 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:58:00.286560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1f4ac36d-8035-4854-9883-7cd0a7215268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:58:00.301977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ac36d-8035-4854-9883-7cd0a7215268 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:58:00.305900 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21d13d-57e1-4e7f-acff-34ad961404e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:58:00.306555 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21d13d-57e1-4e7f-acff-34ad961404e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:00.306940 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21d13d-57e1-4e7f-acff-34ad961404e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:00.307230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21d13d-57e1-4e7f-acff-34ad961404e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:00.315255 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21d13d-57e1-4e7f-acff-34ad961404e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:58:00.316823 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21d13d-57e1-4e7f-acff-34ad961404e6 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:58:00.326029 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1336/2654] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:58:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:58:00.411397 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1170b-d709-44cb-82af-a0da628901cb None None] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:00.411686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1170b-d709-44cb-82af-a0da628901cb None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:00.411962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1170b-d709-44cb-82af-a0da628901cb None None] PATH_INFO: `/` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:00.413402 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1319/2655] 10.222.0.88 () {54 vars in 921 bytes} [Mon Jul 18 16:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 16:58:00.418898 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e75f5f-06f1-4b87-9d27-2cf2ff7a63a6 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:00.419200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e75f5f-06f1-4b87-9d27-2cf2ff7a63a6 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:00.419487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e75f5f-06f1-4b87-9d27-2cf2ff7a63a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:00.439063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b6e75f5f-06f1-4b87-9d27-2cf2ff7a63a6 None None] MFA Rules not processed for user `69ee145599384c6da2cba85a42832dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:58:00.440313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b6e75f5f-06f1-4b87-9d27-2cf2ff7a63a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:58:00.449845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1337/2656] 10.222.0.88 () {58 vars in 1018 bytes} [Mon Jul 18 16:58:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:58:00.460997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1170b-d709-44cb-82af-a0da628901cb None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:58:00.472744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1170b-d709-44cb-82af-a0da628901cb None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:58:00.476371 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7c0d6-5a05-4838-a5fb-66d826df0953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:58:00.477050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7c0d6-5a05-4838-a5fb-66d826df0953 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:00.477333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7c0d6-5a05-4838-a5fb-66d826df0953 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:00.477607 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7c0d6-5a05-4838-a5fb-66d826df0953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:00.490159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7c0d6-5a05-4838-a5fb-66d826df0953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:58:00.490159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7c0d6-5a05-4838-a5fb-66d826df0953 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:58:00.501225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1320/2657] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:58:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:58:08.725217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6d08c-bf41-4185-9d14-0af053f5e7f7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:08.725217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6d08c-bf41-4185-9d14-0af053f5e7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:08.725217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6d08c-bf41-4185-9d14-0af053f5e7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:08.745775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-08e6d08c-bf41-4185-9d14-0af053f5e7f7 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:58:08.747357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-08e6d08c-bf41-4185-9d14-0af053f5e7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:58:08.755432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1338/2658] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 16:58:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:58:08.767456 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7c0d6-5a05-4838-a5fb-66d826df0953 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:58:08.779522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7c0d6-5a05-4838-a5fb-66d826df0953 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:58:08.784910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a3b29-7a2d-47e4-b0dd-f41b1903b38c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:58:08.785582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a3b29-7a2d-47e4-b0dd-f41b1903b38c service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:08.786413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a3b29-7a2d-47e4-b0dd-f41b1903b38c service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:08.786707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a3b29-7a2d-47e4-b0dd-f41b1903b38c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:08.795476 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a3b29-7a2d-47e4-b0dd-f41b1903b38c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:58:08.797084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a3b29-7a2d-47e4-b0dd-f41b1903b38c service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:58:08.808439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1321/2659] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:58:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:58:09.453369 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424b8f5-14ce-4864-a51f-d92e108f1405 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:09.453832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424b8f5-14ce-4864-a51f-d92e108f1405 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:09.456327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424b8f5-14ce-4864-a51f-d92e108f1405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:09.477728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-e424b8f5-14ce-4864-a51f-d92e108f1405 None None] MFA Rules not processed for user `730d6426b4ee4cf38893fe2e60b97230`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:58:09.478903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-e424b8f5-14ce-4864-a51f-d92e108f1405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:58:09.488962 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1339/2660] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:58:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:58:18.802821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=41 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:58:19.471022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=60 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:58:22.020851 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a3b29-7a2d-47e4-b0dd-f41b1903b38c service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:58:22.022375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-124a3b29-7a2d-47e4-b0dd-f41b1903b38c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:58:22.113399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a3b29-7a2d-47e4-b0dd-f41b1903b38c service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:58:22.118712 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621be4e-009d-4751-b51a-95a2fee0dd54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:58:22.119360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621be4e-009d-4751-b51a-95a2fee0dd54 service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:22.119640 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621be4e-009d-4751-b51a-95a2fee0dd54 service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:22.119916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621be4e-009d-4751-b51a-95a2fee0dd54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:22.121741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a621be4e-009d-4751-b51a-95a2fee0dd54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:58:22.133176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621be4e-009d-4751-b51a-95a2fee0dd54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:58:22.134744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621be4e-009d-4751-b51a-95a2fee0dd54 service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:58:22.145351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1322/2661] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:58:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:58:32.140264 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=41 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:58:35.862832 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1c260-c299-403e-864d-7fb1168f9e4b None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:35.863393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1c260-c299-403e-864d-7fb1168f9e4b None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:35.863393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1c260-c299-403e-864d-7fb1168f9e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:35.882656 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f4b1c260-c299-403e-864d-7fb1168f9e4b None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:58:35.883874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f4b1c260-c299-403e-864d-7fb1168f9e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:58:35.891233 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1340/2662] 10.222.0.88 () {58 vars in 967 bytes} [Mon Jul 18 16:58:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:58:35.903639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621be4e-009d-4751-b51a-95a2fee0dd54 service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:58:35.914977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621be4e-009d-4751-b51a-95a2fee0dd54 service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:58:35.920002 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07c11d-869c-4b76-8428-2d015d309f80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:58:35.920642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07c11d-869c-4b76-8428-2d015d309f80 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:58:35.921075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07c11d-869c-4b76-8428-2d015d309f80 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:58:35.921352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07c11d-869c-4b76-8428-2d015d309f80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:58:35.931178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07c11d-869c-4b76-8428-2d015d309f80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:58:35.932769 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07c11d-869c-4b76-8428-2d015d309f80 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:58:35.942443 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1323/2663] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 16:58:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 16:58:45.876802 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:58:45.938031 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=6 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:59:20.842152 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1c260-c299-403e-864d-7fb1168f9e4b None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 16:59:20.843655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f4b1c260-c299-403e-864d-7fb1168f9e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:59:20.944723 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1c260-c299-403e-864d-7fb1168f9e4b None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 16:59:20.949749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75874eba-9cf2-443f-bf10-1416f1d3d4fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 16:59:20.950442 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75874eba-9cf2-443f-bf10-1416f1d3d4fb service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:59:20.950787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75874eba-9cf2-443f-bf10-1416f1d3d4fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:59:20.951610 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75874eba-9cf2-443f-bf10-1416f1d3d4fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:59:20.953208 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-75874eba-9cf2-443f-bf10-1416f1d3d4fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:59:20.966033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75874eba-9cf2-443f-bf10-1416f1d3d4fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 16:59:20.967652 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75874eba-9cf2-443f-bf10-1416f1d3d4fb service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 16:59:20.978678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1341/2664] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 16:59:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 16:59:23.610220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bb746-0bb9-4cf8-b990-d70e345973fe None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:59:23.610802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bb746-0bb9-4cf8-b990-d70e345973fe None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:59:23.610802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bb746-0bb9-4cf8-b990-d70e345973fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:59:23.627948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-710bb746-0bb9-4cf8-b990-d70e345973fe None None] MFA Rules not processed for user `24b3c774c1b04537a970c1e7dfdd6ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:59:23.629052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-710bb746-0bb9-4cf8-b990-d70e345973fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:59:23.688870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1324/2665] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:59:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:59:29.616774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda388b1-36a6-4df0-8cdb-fcc15b367696 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:59:29.617391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda388b1-36a6-4df0-8cdb-fcc15b367696 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:59:29.617391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda388b1-36a6-4df0-8cdb-fcc15b367696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:59:29.636322 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-eda388b1-36a6-4df0-8cdb-fcc15b367696 None None] MFA Rules not processed for user `cd69c5111dac492a9177128317f0962a`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:59:29.637703 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eda388b1-36a6-4df0-8cdb-fcc15b367696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:59:29.647617 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1342/2666] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:59:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:59:31.309978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbc2d2-e923-4bea-a16e-06d0e40ebd83 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 16:59:31.311462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbc2d2-e923-4bea-a16e-06d0e40ebd83 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 16:59:31.311462 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbc2d2-e923-4bea-a16e-06d0e40ebd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 16:59:31.330714 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-dbdbc2d2-e923-4bea-a16e-06d0e40ebd83 None None] MFA Rules not processed for user `730d6426b4ee4cf38893fe2e60b97230`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 16:59:31.338278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-dbdbc2d2-e923-4bea-a16e-06d0e40ebd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 16:59:31.338938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1325/2667] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 16:59:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 16:59:39.630410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=38 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 16:59:41.324356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=28 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:00:18.544170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda388b1-36a6-4df0-8cdb-fcc15b367696 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:00:18.545648 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eda388b1-36a6-4df0-8cdb-fcc15b367696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:18.558395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda388b1-36a6-4df0-8cdb-fcc15b367696 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:00:18.562508 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7400d8-f145-42c0-9878-4a76b2172a49 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54c7869983bf4534b3e730af0185eac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:00:18.563147 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7400d8-f145-42c0-9878-4a76b2172a49 service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:00:18.563436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7400d8-f145-42c0-9878-4a76b2172a49 service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:00:18.563716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7400d8-f145-42c0-9878-4a76b2172a49 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:00:18.565591 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dd7400d8-f145-42c0-9878-4a76b2172a49 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:18.580907 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7400d8-f145-42c0-9878-4a76b2172a49 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:00:18.582355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7400d8-f145-42c0-9878-4a76b2172a49 service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:00:18.593438 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1343/2668] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:00:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:00:18.894304 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbc2d2-e923-4bea-a16e-06d0e40ebd83 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:00:18.914443 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbc2d2-e923-4bea-a16e-06d0e40ebd83 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:00:18.918219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c52c0-0846-4dfd-bd4a-2034779af8e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:00:18.922884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c52c0-0846-4dfd-bd4a-2034779af8e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:00:18.922884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c52c0-0846-4dfd-bd4a-2034779af8e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:00:18.922884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c52c0-0846-4dfd-bd4a-2034779af8e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:00:18.922884 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-173c52c0-0846-4dfd-bd4a-2034779af8e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:18.936366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c52c0-0846-4dfd-bd4a-2034779af8e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:00:18.937866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c52c0-0846-4dfd-bd4a-2034779af8e8 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:00:18.950058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1326/2669] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:00:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:00:28.588526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:00:28.944741 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=10 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:00:37.125071 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260641e-7d75-4d16-8c64-6d9a130077ef None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:00:37.125622 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260641e-7d75-4d16-8c64-6d9a130077ef None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:00:37.126081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260641e-7d75-4d16-8c64-6d9a130077ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:00:37.182718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-a260641e-7d75-4d16-8c64-6d9a130077ef None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:00:37.183952 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-a260641e-7d75-4d16-8c64-6d9a130077ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:37.202063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1344/2670] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 17:00:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:00:37.385222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c52c0-0846-4dfd-bd4a-2034779af8e8 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:00:37.407269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c52c0-0846-4dfd-bd4a-2034779af8e8 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:00:37.412583 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7bf3b-134e-400c-8a49-4735495f8890 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:00:37.413261 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7bf3b-134e-400c-8a49-4735495f8890 service swift] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:00:37.413542 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7bf3b-134e-400c-8a49-4735495f8890 service swift] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:00:37.413817 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb7bf3b-134e-400c-8a49-4735495f8890 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:00:37.422508 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7bf3b-134e-400c-8a49-4735495f8890 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:00:37.424119 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb7bf3b-134e-400c-8a49-4735495f8890 service swift] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:00:37.432667 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1327/2671] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 17:00:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 17:00:47.198751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=18 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:00:47.431164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=10 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:00:52.775063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147807e-da4b-4662-ba43-103a34dc941e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:00:52.775546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147807e-da4b-4662-ba43-103a34dc941e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:00:52.775737 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147807e-da4b-4662-ba43-103a34dc941e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:00:52.795282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f147807e-da4b-4662-ba43-103a34dc941e None None] MFA Rules not processed for user `730d6426b4ee4cf38893fe2e60b97230`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:00:52.796633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f147807e-da4b-4662-ba43-103a34dc941e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:52.803860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1345/2672] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:00:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:00:57.302666 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7bf3b-134e-400c-8a49-4735495f8890 service swift] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:00:57.311020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb7bf3b-134e-400c-8a49-4735495f8890 service swift] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:00:57.315369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf87d6b-1f26-4b82-98c9-4f1d10bb827b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:00:57.315969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf87d6b-1f26-4b82-98c9-4f1d10bb827b service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:00:57.316258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf87d6b-1f26-4b82-98c9-4f1d10bb827b service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:00:57.316544 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf87d6b-1f26-4b82-98c9-4f1d10bb827b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:00:57.318645 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-aaf87d6b-1f26-4b82-98c9-4f1d10bb827b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:57.341437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf87d6b-1f26-4b82-98c9-4f1d10bb827b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:00:57.343482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf87d6b-1f26-4b82-98c9-4f1d10bb827b service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:00:57.351647 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1328/2673] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:00:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:00:58.055641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147807e-da4b-4662-ba43-103a34dc941e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:00:58.075122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147807e-da4b-4662-ba43-103a34dc941e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:00:58.087841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bebb7d-87ba-4180-add7-0ab1235c830a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:00:58.087841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bebb7d-87ba-4180-add7-0ab1235c830a service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:00:58.087841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bebb7d-87ba-4180-add7-0ab1235c830a service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:00:58.087841 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bebb7d-87ba-4180-add7-0ab1235c830a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:00:58.089655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-29bebb7d-87ba-4180-add7-0ab1235c830a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:58.091398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed4dc7-08d9-4302-a82e-853f5d6ec425 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:00:58.092008 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed4dc7-08d9-4302-a82e-853f5d6ec425 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:00:58.093196 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed4dc7-08d9-4302-a82e-853f5d6ec425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:00:58.105674 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bebb7d-87ba-4180-add7-0ab1235c830a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:00:58.107719 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bebb7d-87ba-4180-add7-0ab1235c830a service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:00:58.124249 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-43ed4dc7-08d9-4302-a82e-853f5d6ec425 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:00:58.125869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-43ed4dc7-08d9-4302-a82e-853f5d6ec425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:00:58.126905 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1346/2674] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:00:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:00:58.142132 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1329/2675] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 17:00:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:00:58.161613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bebb7d-87ba-4180-add7-0ab1235c830a service nova] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:00:58.179483 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bebb7d-87ba-4180-add7-0ab1235c830a service nova] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:00:58.186388 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386d009-932a-4152-8b55-088535f0e8e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:00:58.186597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386d009-932a-4152-8b55-088535f0e8e8 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:00:58.187275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386d009-932a-4152-8b55-088535f0e8e8 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:00:58.187567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386d009-932a-4152-8b55-088535f0e8e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:00:58.197920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386d009-932a-4152-8b55-088535f0e8e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:00:58.201316 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5386d009-932a-4152-8b55-088535f0e8e8 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:00:58.211864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1347/2676] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 17:00:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 17:01:08.113250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=14 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:01:08.208260 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=15 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:01:15.126321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f204b-ae38-4794-b177-d4fb9627cbf3 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:15.126829 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f204b-ae38-4794-b177-d4fb9627cbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:15.126926 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f204b-ae38-4794-b177-d4fb9627cbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:15.145720 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-712f204b-ae38-4794-b177-d4fb9627cbf3 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:01:15.156947 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-712f204b-ae38-4794-b177-d4fb9627cbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:15.156947 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1330/2677] 10.222.0.88 () {58 vars in 967 bytes} [Mon Jul 18 17:01:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:01:15.167411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386d009-932a-4152-8b55-088535f0e8e8 service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:15.178921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386d009-932a-4152-8b55-088535f0e8e8 service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:15.184131 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc33f5-1a42-43b0-a1dd-e1602a06ab7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:15.184716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc33f5-1a42-43b0-a1dd-e1602a06ab7f service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:15.185076 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc33f5-1a42-43b0-a1dd-e1602a06ab7f service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:15.185998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc33f5-1a42-43b0-a1dd-e1602a06ab7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:15.195357 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc33f5-1a42-43b0-a1dd-e1602a06ab7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:15.199542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc33f5-1a42-43b0-a1dd-e1602a06ab7f service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:15.205734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1348/2678] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 17:01:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:01:25.140554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:01:25.202371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=6 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:01:49.994251 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f204b-ae38-4794-b177-d4fb9627cbf3 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:50.006334 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f204b-ae38-4794-b177-d4fb9627cbf3 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:50.010117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7608854-3798-4b7b-a2a2-620897fdac92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:50.010955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7608854-3798-4b7b-a2a2-620897fdac92 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:50.011363 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7608854-3798-4b7b-a2a2-620897fdac92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:50.011684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7608854-3798-4b7b-a2a2-620897fdac92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:50.013499 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d7608854-3798-4b7b-a2a2-620897fdac92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:50.051004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7608854-3798-4b7b-a2a2-620897fdac92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:50.052502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7608854-3798-4b7b-a2a2-620897fdac92 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:50.073274 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1331/2679] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:01:50.307391 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc33f5-1a42-43b0-a1dd-e1602a06ab7f service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:50.318420 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc33f5-1a42-43b0-a1dd-e1602a06ab7f service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:50.322422 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e0651-486a-4ed7-b3d6-3ea20367983e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:50.323082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0651-486a-4ed7-b3d6-3ea20367983e service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:50.323363 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0651-486a-4ed7-b3d6-3ea20367983e service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:50.323639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e0651-486a-4ed7-b3d6-3ea20367983e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:50.325910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-959e0651-486a-4ed7-b3d6-3ea20367983e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:50.339214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e0651-486a-4ed7-b3d6-3ea20367983e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:50.340668 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e0651-486a-4ed7-b3d6-3ea20367983e service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:50.350143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1349/2680] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:01:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:01:54.067598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7608854-3798-4b7b-a2a2-620897fdac92 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:54.071068 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d7608854-3798-4b7b-a2a2-620897fdac92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:54.088024 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7608854-3798-4b7b-a2a2-620897fdac92 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:54.094129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32517ba-bd20-4d94-98bd-8f0d105564db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:54.095096 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32517ba-bd20-4d94-98bd-8f0d105564db service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:54.095594 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32517ba-bd20-4d94-98bd-8f0d105564db service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:54.095938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32517ba-bd20-4d94-98bd-8f0d105564db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:54.098051 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c32517ba-bd20-4d94-98bd-8f0d105564db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:54.111846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32517ba-bd20-4d94-98bd-8f0d105564db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:54.115570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32517ba-bd20-4d94-98bd-8f0d105564db service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:54.126044 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1332/2681] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:01:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:01:58.305868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e0651-486a-4ed7-b3d6-3ea20367983e service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:58.307475 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-959e0651-486a-4ed7-b3d6-3ea20367983e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:58.322072 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e0651-486a-4ed7-b3d6-3ea20367983e service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:58.327188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:58.328093 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:58.328384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:58.329034 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] PATH_INFO: `/v3/users/cd69c5111dac492a9177128317f0962a` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:58.329940 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd69c5111dac492a9177128317f0962a)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:58.331992 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:58.507952 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd69c5111dac492a9177128317f0962a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:58.524918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd69c5111dac492a9177128317f0962a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:58.531971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd69c5111dac492a9177128317f0962a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:58.570441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32517ba-bd20-4d94-98bd-8f0d105564db service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:58.580630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32517ba-bd20-4d94-98bd-8f0d105564db service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:58.584347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de478fec-c609-4984-bc3e-935b0d7b97a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:58.585013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de478fec-c609-4984-bc3e-935b0d7b97a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:58.585303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de478fec-c609-4984-bc3e-935b0d7b97a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:58.585581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de478fec-c609-4984-bc3e-935b0d7b97a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:58.587515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-de478fec-c609-4984-bc3e-935b0d7b97a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:58.600539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd69c5111dac492a9177128317f0962a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:58.601046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de478fec-c609-4984-bc3e-935b0d7b97a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:58.601200 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd69c5111dac492a9177128317f0962a'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:58.602300 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de478fec-c609-4984-bc3e-935b0d7b97a5 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:58.607490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1350/2682] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:01:58 2022] DELETE /identity/v3/users/cd69c5111dac492a9177128317f0962a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:01:58.612784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1333/2683] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:01:58.906283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:58.910096 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:59.008123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432bb1b-567d-4548-9bcb-96d499de5e70 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:59.014311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:59.015192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:59.015479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:59.015763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] PATH_INFO: `/v3/projects/5d68154d9fdb40f0848b676c1fed6a50` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:59.017026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d68154d9fdb40f0848b676c1fed6a50)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:59.018983 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:59.149977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d68154d9fdb40f0848b676c1fed6a50', 'request_id': 'req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:59.235033 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d68154d9fdb40f0848b676c1fed6a50', 'request_id': 'req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:59.239938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:01:59.242694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d68154d9fdb40f0848b676c1fed6a50', 'request_id': 'req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:59.254988 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] The token cache is being invalidate because project 5d68154d9fdb40f0848b676c1fed6a50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:01:59.255323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:59.255927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:59.257528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1351/2684] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:01:58 2022] DELETE /identity/v3/projects/5d68154d9fdb40f0848b676c1fed6a50 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:01:59.261982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de478fec-c609-4984-bc3e-935b0d7b97a5 service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:59.272836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de478fec-c609-4984-bc3e-935b0d7b97a5 service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:59.333186 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:59.334220 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:59.334505 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:59.334783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] PATH_INFO: `/v3/users/da3a417ec62d44568f5d4eaaad54f3fa` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:59.336021 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] RBAC: Authorizing `identity:delete_user(user_id=da3a417ec62d44568f5d4eaaad54f3fa)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:59.338103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:01:59.485087 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da3a417ec62d44568f5d4eaaad54f3fa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:59.492680 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da3a417ec62d44568f5d4eaaad54f3fa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:59.506289 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da3a417ec62d44568f5d4eaaad54f3fa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:59.606345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da3a417ec62d44568f5d4eaaad54f3fa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:59.607609 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da3a417ec62d44568f5d4eaaad54f3fa'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:01:59.614528 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1334/2685] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:01:59 2022] DELETE /identity/v3/users/da3a417ec62d44568f5d4eaaad54f3fa => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:01:59.888811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:01:59.899627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:01:59.909113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1af3fc-b83f-45a6-8fdb-ff291c4b019b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:01:59.978989 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:01:59.979866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:01:59.980160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:01:59.980441 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] PATH_INFO: `/v3/projects/13009ce503fe4599ae3cf22798bd7f22` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:01:59.981705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13009ce503fe4599ae3cf22798bd7f22)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:01:59.984443 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:00.094272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13009ce503fe4599ae3cf22798bd7f22', 'request_id': 'req-0d61bd55-94f8-42da-97a4-c207f30fad27'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:00.188687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13009ce503fe4599ae3cf22798bd7f22', 'request_id': 'req-0d61bd55-94f8-42da-97a4-c207f30fad27'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:00.196926 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13009ce503fe4599ae3cf22798bd7f22', 'request_id': 'req-0d61bd55-94f8-42da-97a4-c207f30fad27'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:00.211736 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] The token cache is being invalidate because project 13009ce503fe4599ae3cf22798bd7f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:00.212032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:00.212856 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:00.214561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1352/2686] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:01:59 2022] DELETE /identity/v3/projects/13009ce503fe4599ae3cf22798bd7f22 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:02:02.061943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:02.063485 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:02.077809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:02.134742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b2945-5be0-4adc-a971-8999f90ad61a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:02.195028 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b03d4-ef6d-4b4d-84ba-a0b82a2917d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:02.195682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b03d4-ef6d-4b4d-84ba-a0b82a2917d4 service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:02.195959 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b03d4-ef6d-4b4d-84ba-a0b82a2917d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:02.196245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b03d4-ef6d-4b4d-84ba-a0b82a2917d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:02.198078 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-011b03d4-ef6d-4b4d-84ba-a0b82a2917d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:02.210253 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b03d4-ef6d-4b4d-84ba-a0b82a2917d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:02.211858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b03d4-ef6d-4b4d-84ba-a0b82a2917d4 service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:02.223394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1335/2687] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:02.230262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:02.231749 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:02.304428 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61bd55-94f8-42da-97a4-c207f30fad27 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:02.309060 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a324e9-57f1-45da-aab5-4c7f34e0c9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:02.309932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a324e9-57f1-45da-aab5-4c7f34e0c9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:02.310243 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a324e9-57f1-45da-aab5-4c7f34e0c9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:02.310539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a324e9-57f1-45da-aab5-4c7f34e0c9e7 admin admin] PATH_INFO: `/v3/users/24b3c774c1b04537a970c1e7dfdd6ac3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:02.313203 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a324e9-57f1-45da-aab5-4c7f34e0c9e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b3c774c1b04537a970c1e7dfdd6ac3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:02.313733 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a324e9-57f1-45da-aab5-4c7f34e0c9e7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:02.336343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:02.501625 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05a324e9-57f1-45da-aab5-4c7f34e0c9e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b3c774c1b04537a970c1e7dfdd6ac3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:02.516005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05a324e9-57f1-45da-aab5-4c7f34e0c9e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b3c774c1b04537a970c1e7dfdd6ac3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:02.523159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05a324e9-57f1-45da-aab5-4c7f34e0c9e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b3c774c1b04537a970c1e7dfdd6ac3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:02.583467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05a324e9-57f1-45da-aab5-4c7f34e0c9e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b3c774c1b04537a970c1e7dfdd6ac3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:02.584176 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-05a324e9-57f1-45da-aab5-4c7f34e0c9e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b3c774c1b04537a970c1e7dfdd6ac3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:02.590805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1353/2688] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:02 2022] DELETE /identity/v3/users/24b3c774c1b04537a970c1e7dfdd6ac3 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:02.842055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b03d4-ef6d-4b4d-84ba-a0b82a2917d4 service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:02.853115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b03d4-ef6d-4b4d-84ba-a0b82a2917d4 service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:02.920039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:02.920999 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:02.921244 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:02.921517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] PATH_INFO: `/v3/projects/034f143ba31148c0939a06296ec18a96` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:02.922727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=034f143ba31148c0939a06296ec18a96)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:02.924656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:03.045388 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:03.048278 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '034f143ba31148c0939a06296ec18a96', 'request_id': 'req-ee06a0a7-9615-497c-9ee0-8618bd7133dd'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:03.054821 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '034f143ba31148c0939a06296ec18a96', 'request_id': 'req-ee06a0a7-9615-497c-9ee0-8618bd7133dd'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:03.060642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '034f143ba31148c0939a06296ec18a96', 'request_id': 'req-ee06a0a7-9615-497c-9ee0-8618bd7133dd'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:03.195815 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] The token cache is being invalidate because project 034f143ba31148c0939a06296ec18a96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:03.196488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:03.196668 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:03.198269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1336/2689] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:02 2022] DELETE /identity/v3/projects/034f143ba31148c0939a06296ec18a96 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:03.204632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36129d32-20bc-4de5-b550-4acd94e3f68c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:03.205131 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36129d32-20bc-4de5-b550-4acd94e3f68c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:03.205536 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36129d32-20bc-4de5-b550-4acd94e3f68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:03.252601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-36129d32-20bc-4de5-b550-4acd94e3f68c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:03.254035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-36129d32-20bc-4de5-b550-4acd94e3f68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:03.305349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:03.306763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:03.317545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1354/2690] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:03.324965 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56876e-6920-42f8-a278-7506efeb64bf None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:03.325266 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56876e-6920-42f8-a278-7506efeb64bf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:03.325543 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56876e-6920-42f8-a278-7506efeb64bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:03.344515 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-df56876e-6920-42f8-a278-7506efeb64bf None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:03.345759 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-df56876e-6920-42f8-a278-7506efeb64bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:03.353728 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1355/2691] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:03.357910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56876e-6920-42f8-a278-7506efeb64bf None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:03.367189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56876e-6920-42f8-a278-7506efeb64bf None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:03.369497 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06a0a7-9615-497c-9ee0-8618bd7133dd admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:03.372042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58428f-d80c-4497-8013-25b8e9126571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:03.372702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58428f-d80c-4497-8013-25b8e9126571 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:03.373015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58428f-d80c-4497-8013-25b8e9126571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:03.373295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58428f-d80c-4497-8013-25b8e9126571 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:03.373707 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe704fb-4ba6-47c3-ba59-7fd676ecaf6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:03.374024 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58428f-d80c-4497-8013-25b8e9126571 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:02:03.374331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58428f-d80c-4497-8013-25b8e9126571 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:03.374454 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe704fb-4ba6-47c3-ba59-7fd676ecaf6d service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:03.374635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe704fb-4ba6-47c3-ba59-7fd676ecaf6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:03.374918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe704fb-4ba6-47c3-ba59-7fd676ecaf6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:03.376000 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58428f-d80c-4497-8013-25b8e9126571 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:03.377260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8fe704fb-4ba6-47c3-ba59-7fd676ecaf6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:03.389220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1356/2692] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 17:02:03.393446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58428f-d80c-4497-8013-25b8e9126571 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:03.403422 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58428f-d80c-4497-8013-25b8e9126571 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:03.408866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2593d-2816-4970-82a3-d127a36d354b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:03.409546 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2593d-2816-4970-82a3-d127a36d354b admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:03.409834 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2593d-2816-4970-82a3-d127a36d354b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:03.410271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2593d-2816-4970-82a3-d127a36d354b admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:03.410977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2593d-2816-4970-82a3-d127a36d354b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:03.413046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2593d-2816-4970-82a3-d127a36d354b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:03.441126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe704fb-4ba6-47c3-ba59-7fd676ecaf6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:03.442614 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe704fb-4ba6-47c3-ba59-7fd676ecaf6d service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:03.452482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1337/2693] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:02:03.528803 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1357/2694] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:03 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 17:02:03.532052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe704fb-4ba6-47c3-ba59-7fd676ecaf6d service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:03.543421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe704fb-4ba6-47c3-ba59-7fd676ecaf6d service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:03.602284 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e24c03-b63c-4a80-8f25-9382e1be823e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:03.602957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24c03-b63c-4a80-8f25-9382e1be823e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:03.603241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24c03-b63c-4a80-8f25-9382e1be823e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:03.603520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24c03-b63c-4a80-8f25-9382e1be823e admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:03.604219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e24c03-b63c-4a80-8f25-9382e1be823e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:03.606159 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e24c03-b63c-4a80-8f25-9382e1be823e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:03.669255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1338/2695] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:03 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:03.673406 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2593d-2816-4970-82a3-d127a36d354b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:03.683631 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2593d-2816-4970-82a3-d127a36d354b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:03.688798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02358e-2ae0-4c38-9b73-ba4f21d7164d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:03.689484 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02358e-2ae0-4c38-9b73-ba4f21d7164d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:03.689765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02358e-2ae0-4c38-9b73-ba4f21d7164d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:03.690073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02358e-2ae0-4c38-9b73-ba4f21d7164d admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:03.691052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02358e-2ae0-4c38-9b73-ba4f21d7164d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:03.692632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02358e-2ae0-4c38-9b73-ba4f21d7164d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:03.703677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1358/2696] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:03.707353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e24c03-b63c-4a80-8f25-9382e1be823e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:03.717849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e24c03-b63c-4a80-8f25-9382e1be823e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:03.722558 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9339be3-b758-4743-8127-601b85bb3cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:03.723303 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9339be3-b758-4743-8127-601b85bb3cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:03.723479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9339be3-b758-4743-8127-601b85bb3cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:03.723753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9339be3-b758-4743-8127-601b85bb3cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:03.724479 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9339be3-b758-4743-8127-601b85bb3cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:03.726110 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9339be3-b758-4743-8127-601b85bb3cf9 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:03.736262 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1339/2697] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:03.740389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02358e-2ae0-4c38-9b73-ba4f21d7164d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:03.751310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02358e-2ae0-4c38-9b73-ba4f21d7164d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:03.756031 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c113c4-db1d-491f-8f36-9c0cf5232930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:03.756938 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c113c4-db1d-491f-8f36-9c0cf5232930 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:03.757229 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c113c4-db1d-491f-8f36-9c0cf5232930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:03.757504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c113c4-db1d-491f-8f36-9c0cf5232930 admin admin] PATH_INFO: `/v3/projects/f0b6282ccd52404286aa7442e78e8086/users/9c39b1e3315348799c5bd93c8b0535fd/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:03.770606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:03.773827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c113c4-db1d-491f-8f36-9c0cf5232930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b6282ccd52404286aa7442e78e8086, user_id=9c39b1e3315348799c5bd93c8b0535fd, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:03.776470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c113c4-db1d-491f-8f36-9c0cf5232930 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:03.823295 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1359/2698] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:03 2022] PUT /identity/v3/projects/f0b6282ccd52404286aa7442e78e8086/users/9c39b1e3315348799c5bd93c8b0535fd/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:03.834593 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b738c1-8b72-4e5f-8b57-03175c9cf11b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:03.834890 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b738c1-8b72-4e5f-8b57-03175c9cf11b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:03.835172 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b738c1-8b72-4e5f-8b57-03175c9cf11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:03.847744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:03.863805 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-66b738c1-8b72-4e5f-8b57-03175c9cf11b None None] MFA Rules not processed for user `9c39b1e3315348799c5bd93c8b0535fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:03.865069 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-66b738c1-8b72-4e5f-8b57-03175c9cf11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:03.935863 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1340/2699] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:02:03.947487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c113c4-db1d-491f-8f36-9c0cf5232930 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:03.949305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-03c113c4-db1d-491f-8f36-9c0cf5232930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:04.028308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c113c4-db1d-491f-8f36-9c0cf5232930 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:04.034059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddafea-8753-4081-86f4-6336f97d04e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:04.034716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddafea-8753-4081-86f4-6336f97d04e8 service nova] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:04.035004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddafea-8753-4081-86f4-6336f97d04e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:04.035285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddafea-8753-4081-86f4-6336f97d04e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:04.043125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddafea-8753-4081-86f4-6336f97d04e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:04.045311 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddafea-8753-4081-86f4-6336f97d04e8 service nova] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:04.054283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1360/2700] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:13.193958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b738c1-8b72-4e5f-8b57-03175c9cf11b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:13.266066 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b738c1-8b72-4e5f-8b57-03175c9cf11b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:13.271230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:13.272103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:13.272382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:13.272659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] PATH_INFO: `/v3/users/9c39b1e3315348799c5bd93c8b0535fd` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:13.273971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c39b1e3315348799c5bd93c8b0535fd)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:13.275977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:13.435247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c39b1e3315348799c5bd93c8b0535fd'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:13.444524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c39b1e3315348799c5bd93c8b0535fd'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:13.463687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c39b1e3315348799c5bd93c8b0535fd'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:13.532060 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c39b1e3315348799c5bd93c8b0535fd'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:13.532701 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c39b1e3315348799c5bd93c8b0535fd'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:13.543617 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1341/2701] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:13 2022] DELETE /identity/v3/users/9c39b1e3315348799c5bd93c8b0535fd => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:13.546503 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a337-3636-4887-a0f9-3823d80bdd28 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:13.547084 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a337-3636-4887-a0f9-3823d80bdd28 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:13.548328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140a337-3636-4887-a0f9-3823d80bdd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:13.572608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-f140a337-3636-4887-a0f9-3823d80bdd28 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:13.573887 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f140a337-3636-4887-a0f9-3823d80bdd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:13.648694 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1361/2702] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:13.659601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:13.661090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:13.680052 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:13.731832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e659f-17c5-4c6c-bd7d-a319d621312a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:13.736202 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b7279-739e-4819-ac84-2d822e7ed491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:13.736792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b7279-739e-4819-ac84-2d822e7ed491 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:13.737534 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b7279-739e-4819-ac84-2d822e7ed491 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:13.738225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b7279-739e-4819-ac84-2d822e7ed491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:13.749418 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b7279-739e-4819-ac84-2d822e7ed491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:13.751004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b7279-739e-4819-ac84-2d822e7ed491 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:13.761252 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1342/2703] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:14.145845 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140a337-3636-4887-a0f9-3823d80bdd28 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:14.146399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-f140a337-3636-4887-a0f9-3823d80bdd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:14.164978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140a337-3636-4887-a0f9-3823d80bdd28 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:14.169630 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:14.171123 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:14.171403 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:14.171680 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] PATH_INFO: `/v3/projects/f0b6282ccd52404286aa7442e78e8086` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:14.172890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b6282ccd52404286aa7442e78e8086)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:14.174813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:14.324529 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b6282ccd52404286aa7442e78e8086', 'request_id': 'req-3ce0b43a-79e7-48e6-950d-bdfae624f82d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:14.394840 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b6282ccd52404286aa7442e78e8086', 'request_id': 'req-3ce0b43a-79e7-48e6-950d-bdfae624f82d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:14.401847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b6282ccd52404286aa7442e78e8086', 'request_id': 'req-3ce0b43a-79e7-48e6-950d-bdfae624f82d'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:14.411386 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:14.417512 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] The token cache is being invalidate because project f0b6282ccd52404286aa7442e78e8086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:14.417820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:14.420120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-3ce0b43a-79e7-48e6-950d-bdfae624f82d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:14.422516 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1362/2704] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:14 2022] DELETE /identity/v3/projects/f0b6282ccd52404286aa7442e78e8086 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:14.427674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f595c-e6fb-4048-86e5-1764bc1d6bb4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:14.428538 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f595c-e6fb-4048-86e5-1764bc1d6bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:14.428816 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f595c-e6fb-4048-86e5-1764bc1d6bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:14.451733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-904f595c-e6fb-4048-86e5-1764bc1d6bb4 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:14.453027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-904f595c-e6fb-4048-86e5-1764bc1d6bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:14.516838 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1343/2705] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:14.525206 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c175f5e-26f5-4894-9a8b-80762e0d24f7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:14.525498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c175f5e-26f5-4894-9a8b-80762e0d24f7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:14.525785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c175f5e-26f5-4894-9a8b-80762e0d24f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:14.544650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-6c175f5e-26f5-4894-9a8b-80762e0d24f7 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:14.545945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-6c175f5e-26f5-4894-9a8b-80762e0d24f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:14.554056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1363/2706] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:14.558314 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f595c-e6fb-4048-86e5-1764bc1d6bb4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:14.568805 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f595c-e6fb-4048-86e5-1764bc1d6bb4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:14.574504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03974b-cee0-430b-8eda-85543dc8c65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:14.582488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03974b-cee0-430b-8eda-85543dc8c65a admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:14.582488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03974b-cee0-430b-8eda-85543dc8c65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:14.582488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03974b-cee0-430b-8eda-85543dc8c65a admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:14.582488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03974b-cee0-430b-8eda-85543dc8c65a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:02:14.582488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03974b-cee0-430b-8eda-85543dc8c65a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:14.582488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03974b-cee0-430b-8eda-85543dc8c65a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:14.595318 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1344/2707] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:02:14.596578 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c175f5e-26f5-4894-9a8b-80762e0d24f7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:14.607712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c175f5e-26f5-4894-9a8b-80762e0d24f7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:14.612795 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24001d-7cca-47a2-9e16-1fb784262de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:14.613527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24001d-7cca-47a2-9e16-1fb784262de7 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:14.613813 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24001d-7cca-47a2-9e16-1fb784262de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:14.614092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24001d-7cca-47a2-9e16-1fb784262de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:14.614796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24001d-7cca-47a2-9e16-1fb784262de7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:14.616794 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24001d-7cca-47a2-9e16-1fb784262de7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:14.699319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1364/2708] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:14 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:14.704143 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03974b-cee0-430b-8eda-85543dc8c65a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:14.715152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03974b-cee0-430b-8eda-85543dc8c65a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:14.772540 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c78591-7547-402e-b766-a3e2ed0b7953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:14.773245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c78591-7547-402e-b766-a3e2ed0b7953 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:14.773529 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c78591-7547-402e-b766-a3e2ed0b7953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:14.773803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c78591-7547-402e-b766-a3e2ed0b7953 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:14.774530 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c78591-7547-402e-b766-a3e2ed0b7953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:14.776523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c78591-7547-402e-b766-a3e2ed0b7953 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:14.852151 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1345/2709] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:14 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:14.856409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24001d-7cca-47a2-9e16-1fb784262de7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:14.867262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24001d-7cca-47a2-9e16-1fb784262de7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:14.871978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642f72d-8dd5-4547-8399-baecb61aad31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:14.872696 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642f72d-8dd5-4547-8399-baecb61aad31 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:14.873521 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642f72d-8dd5-4547-8399-baecb61aad31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:14.873812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642f72d-8dd5-4547-8399-baecb61aad31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:14.874547 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642f72d-8dd5-4547-8399-baecb61aad31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:14.876153 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642f72d-8dd5-4547-8399-baecb61aad31 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:14.887849 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1365/2710] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:14.892477 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c78591-7547-402e-b766-a3e2ed0b7953 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:14.903094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c78591-7547-402e-b766-a3e2ed0b7953 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:14.907955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fe600-1ffa-4596-acce-e6d397f70530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:14.908617 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fe600-1ffa-4596-acce-e6d397f70530 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:14.908934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fe600-1ffa-4596-acce-e6d397f70530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:14.909216 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fe600-1ffa-4596-acce-e6d397f70530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:14.909953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fe600-1ffa-4596-acce-e6d397f70530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:14.911567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fe600-1ffa-4596-acce-e6d397f70530 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:14.921763 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1346/2711] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:14.925927 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642f72d-8dd5-4547-8399-baecb61aad31 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:14.936848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642f72d-8dd5-4547-8399-baecb61aad31 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:14.941761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c993dd-9772-4844-950a-b03fd2ca0813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:14.942639 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c993dd-9772-4844-950a-b03fd2ca0813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:14.942922 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c993dd-9772-4844-950a-b03fd2ca0813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:14.943196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c993dd-9772-4844-950a-b03fd2ca0813 admin admin] PATH_INFO: `/v3/projects/46aa77f9a02243ca8585965d3d7863c1/users/bc3962d365a74ae68ff0b10b8830688b/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:14.958517 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c993dd-9772-4844-950a-b03fd2ca0813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46aa77f9a02243ca8585965d3d7863c1, user_id=bc3962d365a74ae68ff0b10b8830688b, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:14.961207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c993dd-9772-4844-950a-b03fd2ca0813 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:15.069734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1366/2712] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:14 2022] PUT /identity/v3/projects/46aa77f9a02243ca8585965d3d7863c1/users/bc3962d365a74ae68ff0b10b8830688b/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:15.080150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16685e1-728f-4c7c-bec3-e44d021455f7 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:15.080490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16685e1-728f-4c7c-bec3-e44d021455f7 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:15.080775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16685e1-728f-4c7c-bec3-e44d021455f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:15.108893 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e16685e1-728f-4c7c-bec3-e44d021455f7 None None] MFA Rules not processed for user `bc3962d365a74ae68ff0b10b8830688b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:15.110145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e16685e1-728f-4c7c-bec3-e44d021455f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:15.121891 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:15.175242 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1347/2713] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:15.181331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c993dd-9772-4844-950a-b03fd2ca0813 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:15.244101 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c993dd-9772-4844-950a-b03fd2ca0813 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:15.249419 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748b378-203b-4eab-a11c-83607b6377ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:15.250073 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748b378-203b-4eab-a11c-83607b6377ed admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:15.250353 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748b378-203b-4eab-a11c-83607b6377ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:15.250626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748b378-203b-4eab-a11c-83607b6377ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:15.251309 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748b378-203b-4eab-a11c-83607b6377ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:15.253230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748b378-203b-4eab-a11c-83607b6377ed admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:15.356343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1367/2714] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:15 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:15.360958 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16685e1-728f-4c7c-bec3-e44d021455f7 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:15.372500 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16685e1-728f-4c7c-bec3-e44d021455f7 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:15.441389 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bcaab-8bc9-4169-a9c5-9d3b2cf50b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:15.442058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bcaab-8bc9-4169-a9c5-9d3b2cf50b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:15.442332 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bcaab-8bc9-4169-a9c5-9d3b2cf50b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:15.442606 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bcaab-8bc9-4169-a9c5-9d3b2cf50b2a admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:15.443295 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bcaab-8bc9-4169-a9c5-9d3b2cf50b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:15.445241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bcaab-8bc9-4169-a9c5-9d3b2cf50b2a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:15.562368 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1348/2715] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:15 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:15.566621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748b378-203b-4eab-a11c-83607b6377ed admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:15.579196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748b378-203b-4eab-a11c-83607b6377ed admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:15.584398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fe6f3-9ee0-40eb-955f-d3f284d3bf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:15.585207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fe6f3-9ee0-40eb-955f-d3f284d3bf39 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:15.585557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fe6f3-9ee0-40eb-955f-d3f284d3bf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:15.585914 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fe6f3-9ee0-40eb-955f-d3f284d3bf39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:15.586713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33fe6f3-9ee0-40eb-955f-d3f284d3bf39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:15.588462 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33fe6f3-9ee0-40eb-955f-d3f284d3bf39 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:15.599987 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1368/2716] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:15.603859 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bcaab-8bc9-4169-a9c5-9d3b2cf50b2a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:15.614610 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bcaab-8bc9-4169-a9c5-9d3b2cf50b2a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:15.619359 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb87199-3518-43a9-9dc5-dbcc6cd4745e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:15.620036 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb87199-3518-43a9-9dc5-dbcc6cd4745e admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:15.620320 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb87199-3518-43a9-9dc5-dbcc6cd4745e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:15.620619 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb87199-3518-43a9-9dc5-dbcc6cd4745e admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:15.621446 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb87199-3518-43a9-9dc5-dbcc6cd4745e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:15.623877 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb87199-3518-43a9-9dc5-dbcc6cd4745e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:15.634406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1349/2717] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:15.639515 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fe6f3-9ee0-40eb-955f-d3f284d3bf39 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:15.649490 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fe6f3-9ee0-40eb-955f-d3f284d3bf39 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:15.654143 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d371c-45f5-4900-8210-43021d50878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:15.655086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d371c-45f5-4900-8210-43021d50878b admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:15.655399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d371c-45f5-4900-8210-43021d50878b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:15.655686 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d371c-45f5-4900-8210-43021d50878b admin admin] PATH_INFO: `/v3/projects/ba5f34946323420f9819d5f51cb41a57/users/02a87ca3f9e94b98bd0038c977d2d7fb/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:15.670309 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d371c-45f5-4900-8210-43021d50878b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba5f34946323420f9819d5f51cb41a57, user_id=02a87ca3f9e94b98bd0038c977d2d7fb, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:15.673015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d371c-45f5-4900-8210-43021d50878b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:15.772459 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1369/2718] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:15 2022] PUT /identity/v3/projects/ba5f34946323420f9819d5f51cb41a57/users/02a87ca3f9e94b98bd0038c977d2d7fb/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:15.784237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b61e8-8aba-445a-86cf-3f0e80061d67 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:15.784543 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b61e8-8aba-445a-86cf-3f0e80061d67 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:15.784870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b61e8-8aba-445a-86cf-3f0e80061d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:15.813103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ef5b61e8-8aba-445a-86cf-3f0e80061d67 None None] MFA Rules not processed for user `02a87ca3f9e94b98bd0038c977d2d7fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:15.814259 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ef5b61e8-8aba-445a-86cf-3f0e80061d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:15.881436 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1350/2719] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:15.885868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d371c-45f5-4900-8210-43021d50878b admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:15.945624 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:15.960337 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d371c-45f5-4900-8210-43021d50878b admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:15.965088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d233a9-4703-4e0f-86ad-db58e29ba9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:15.965761 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d233a9-4703-4e0f-86ad-db58e29ba9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:15.966049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d233a9-4703-4e0f-86ad-db58e29ba9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:15.966328 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d233a9-4703-4e0f-86ad-db58e29ba9f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:15.967054 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d233a9-4703-4e0f-86ad-db58e29ba9f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:15.969008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d233a9-4703-4e0f-86ad-db58e29ba9f2 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:16.104974 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1370/2720] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:15 2022] POST /identity/v3/projects => generated 387 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:16.110309 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b61e8-8aba-445a-86cf-3f0e80061d67 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:16.121637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b61e8-8aba-445a-86cf-3f0e80061d67 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:16.140533 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:16.182092 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939a292-506c-4d6c-a6e1-a70d3021e27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:16.186688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939a292-506c-4d6c-a6e1-a70d3021e27a admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:16.186688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939a292-506c-4d6c-a6e1-a70d3021e27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:16.186688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939a292-506c-4d6c-a6e1-a70d3021e27a admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:16.186688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939a292-506c-4d6c-a6e1-a70d3021e27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:16.186688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e939a292-506c-4d6c-a6e1-a70d3021e27a admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:16.332732 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1351/2721] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:16 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:16.337101 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d233a9-4703-4e0f-86ad-db58e29ba9f2 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:16.347787 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d233a9-4703-4e0f-86ad-db58e29ba9f2 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:16.352249 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71a720-68e0-49e9-9b87-db73578a3fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:16.352930 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71a720-68e0-49e9-9b87-db73578a3fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:16.353240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71a720-68e0-49e9-9b87-db73578a3fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:16.353517 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71a720-68e0-49e9-9b87-db73578a3fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:16.354241 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71a720-68e0-49e9-9b87-db73578a3fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:16.355885 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71a720-68e0-49e9-9b87-db73578a3fa4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:16.366432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1371/2722] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:16.370674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939a292-506c-4d6c-a6e1-a70d3021e27a admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:16.381228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939a292-506c-4d6c-a6e1-a70d3021e27a admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:16.385952 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec1152-9c1e-46c7-a65b-39b25f3266ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:16.386834 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec1152-9c1e-46c7-a65b-39b25f3266ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:16.387226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec1152-9c1e-46c7-a65b-39b25f3266ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:16.387395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec1152-9c1e-46c7-a65b-39b25f3266ae admin admin] PATH_INFO: `/v3/projects/46217432938943328ae4a9b2c053340b/users/cf90906a38024d05aeabb03d29079c90/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:16.403029 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec1152-9c1e-46c7-a65b-39b25f3266ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=46217432938943328ae4a9b2c053340b, user_id=cf90906a38024d05aeabb03d29079c90, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:16.405725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec1152-9c1e-46c7-a65b-39b25f3266ae admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:16.498800 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1352/2723] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:16 2022] PUT /identity/v3/projects/46217432938943328ae4a9b2c053340b/users/cf90906a38024d05aeabb03d29079c90/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:16.503380 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71a720-68e0-49e9-9b87-db73578a3fa4 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:16.514560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71a720-68e0-49e9-9b87-db73578a3fa4 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:16.572593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfedd49-5826-4ba4-b252-3101d3fa95ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:16.573352 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfedd49-5826-4ba4-b252-3101d3fa95ab admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:16.573598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfedd49-5826-4ba4-b252-3101d3fa95ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:16.573876 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfedd49-5826-4ba4-b252-3101d3fa95ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:16.574607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfedd49-5826-4ba4-b252-3101d3fa95ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:16.576330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfedd49-5826-4ba4-b252-3101d3fa95ab admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:16.586460 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1372/2724] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:16.590601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec1152-9c1e-46c7-a65b-39b25f3266ae admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:16.601620 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec1152-9c1e-46c7-a65b-39b25f3266ae admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:16.609390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7857c40-6445-4195-bbdc-4093d3fe9d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:16.609390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7857c40-6445-4195-bbdc-4093d3fe9d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:16.609390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7857c40-6445-4195-bbdc-4093d3fe9d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:16.609390 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7857c40-6445-4195-bbdc-4093d3fe9d2d admin admin] PATH_INFO: `/v3/projects/46217432938943328ae4a9b2c053340b/users/cf90906a38024d05aeabb03d29079c90/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:16.610531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7857c40-6445-4195-bbdc-4093d3fe9d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=46217432938943328ae4a9b2c053340b, user_id=cf90906a38024d05aeabb03d29079c90, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:16.614158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7857c40-6445-4195-bbdc-4093d3fe9d2d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:16.629871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1353/2725] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:16 2022] PUT /identity/v3/projects/46217432938943328ae4a9b2c053340b/users/cf90906a38024d05aeabb03d29079c90/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:16.646752 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca7c6b-78b9-4380-a3ee-1553ed540fcf None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:16.647107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca7c6b-78b9-4380-a3ee-1553ed540fcf None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:16.647398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca7c6b-78b9-4380-a3ee-1553ed540fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:16.675098 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c7ca7c6b-78b9-4380-a3ee-1553ed540fcf None None] MFA Rules not processed for user `cf90906a38024d05aeabb03d29079c90`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:16.676310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c7ca7c6b-78b9-4380-a3ee-1553ed540fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:16.745720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1373/2726] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:16.757122 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7857c40-6445-4195-bbdc-4093d3fe9d2d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:16.759369 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-f7857c40-6445-4195-bbdc-4093d3fe9d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:16.817233 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7857c40-6445-4195-bbdc-4093d3fe9d2d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:16.820906 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6a3dd-43d9-46f3-9da4-4c26b40705f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:16.821546 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6a3dd-43d9-46f3-9da4-4c26b40705f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:16.821828 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6a3dd-43d9-46f3-9da4-4c26b40705f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:16.822102 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6a3dd-43d9-46f3-9da4-4c26b40705f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:16.831272 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6a3dd-43d9-46f3-9da4-4c26b40705f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:16.832701 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6a3dd-43d9-46f3-9da4-4c26b40705f4 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:16.842365 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1354/2727] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:17.880332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca7c6b-78b9-4380-a3ee-1553ed540fcf None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:17.881852 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c7ca7c6b-78b9-4380-a3ee-1553ed540fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:17.894127 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca7c6b-78b9-4380-a3ee-1553ed540fcf None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:17.897913 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee9015-a705-4930-adb5-41c32cafc465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:17.898556 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee9015-a705-4930-adb5-41c32cafc465 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:17.898864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee9015-a705-4930-adb5-41c32cafc465 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:17.899281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee9015-a705-4930-adb5-41c32cafc465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:17.942066 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:17.958194 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee9015-a705-4930-adb5-41c32cafc465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:17.959675 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee9015-a705-4930-adb5-41c32cafc465 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:17.969071 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1374/2728] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:20.295502 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6a3dd-43d9-46f3-9da4-4c26b40705f4 service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:20.297817 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d2e6a3dd-43d9-46f3-9da4-4c26b40705f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:20.369392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6a3dd-43d9-46f3-9da4-4c26b40705f4 service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:20.376150 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41013eb-f183-49be-b8de-ae5aea6354ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:20.376848 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41013eb-f183-49be-b8de-ae5aea6354ea service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:20.377163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41013eb-f183-49be-b8de-ae5aea6354ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:20.377438 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41013eb-f183-49be-b8de-ae5aea6354ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:20.379449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a41013eb-f183-49be-b8de-ae5aea6354ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:20.412056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:20.445373 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41013eb-f183-49be-b8de-ae5aea6354ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:20.446836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41013eb-f183-49be-b8de-ae5aea6354ea service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:20.456779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1355/2729] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:22.889360 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee9015-a705-4930-adb5-41c32cafc465 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:22.899126 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee9015-a705-4930-adb5-41c32cafc465 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:22.903012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca85788-e123-485a-8f66-94e461adb40c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:22.903747 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca85788-e123-485a-8f66-94e461adb40c service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:22.904134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca85788-e123-485a-8f66-94e461adb40c service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:22.904502 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca85788-e123-485a-8f66-94e461adb40c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:22.975377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca85788-e123-485a-8f66-94e461adb40c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:22.979682 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca85788-e123-485a-8f66-94e461adb40c service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:22.991693 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1375/2730] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:24.269446 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41013eb-f183-49be-b8de-ae5aea6354ea service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:24.271527 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a41013eb-f183-49be-b8de-ae5aea6354ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:24.287201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41013eb-f183-49be-b8de-ae5aea6354ea service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:24.292988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c9669-e6bb-4775-bcdb-fb7d009d4a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:24.293449 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c9669-e6bb-4775-bcdb-fb7d009d4a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:24.293732 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c9669-e6bb-4775-bcdb-fb7d009d4a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:24.294007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c9669-e6bb-4775-bcdb-fb7d009d4a48 admin admin] PATH_INFO: `/v3/users/730d6426b4ee4cf38893fe2e60b97230` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:24.295232 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c9669-e6bb-4775-bcdb-fb7d009d4a48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=730d6426b4ee4cf38893fe2e60b97230)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:24.297350 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c9669-e6bb-4775-bcdb-fb7d009d4a48 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:24.488144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8a8c9669-e6bb-4775-bcdb-fb7d009d4a48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '730d6426b4ee4cf38893fe2e60b97230'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:24.495809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8a8c9669-e6bb-4775-bcdb-fb7d009d4a48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '730d6426b4ee4cf38893fe2e60b97230'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:24.509413 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8a8c9669-e6bb-4775-bcdb-fb7d009d4a48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '730d6426b4ee4cf38893fe2e60b97230'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:24.588787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8a8c9669-e6bb-4775-bcdb-fb7d009d4a48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '730d6426b4ee4cf38893fe2e60b97230'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:24.589480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-8a8c9669-e6bb-4775-bcdb-fb7d009d4a48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '730d6426b4ee4cf38893fe2e60b97230'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:24.596592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1356/2731] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:24 2022] DELETE /identity/v3/users/730d6426b4ee4cf38893fe2e60b97230 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:24.875912 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca85788-e123-485a-8f66-94e461adb40c service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:24.893516 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca85788-e123-485a-8f66-94e461adb40c service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:24.955539 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:24.956409 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:24.956699 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:24.957026 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] PATH_INFO: `/v3/projects/25de8ee0667645a3864d846675d460e1` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:24.958246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25de8ee0667645a3864d846675d460e1)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:24.960158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:25.058553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25de8ee0667645a3864d846675d460e1', 'request_id': 'req-c4c1fca0-ed56-4926-a588-9a90917ee2f4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:25.138022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25de8ee0667645a3864d846675d460e1', 'request_id': 'req-c4c1fca0-ed56-4926-a588-9a90917ee2f4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:25.144755 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25de8ee0667645a3864d846675d460e1', 'request_id': 'req-c4c1fca0-ed56-4926-a588-9a90917ee2f4'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:25.156059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] The token cache is being invalidate because project 25de8ee0667645a3864d846675d460e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:25.156348 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:25.157021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-c4c1fca0-ed56-4926-a588-9a90917ee2f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:25.158560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1376/2732] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:24 2022] DELETE /identity/v3/projects/25de8ee0667645a3864d846675d460e1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:25.163483 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fd07e-55ea-4544-b51a-97ee6a10bd1a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:25.163776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fd07e-55ea-4544-b51a-97ee6a10bd1a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:25.164059 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fd07e-55ea-4544-b51a-97ee6a10bd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:25.181829 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d13fd07e-55ea-4544-b51a-97ee6a10bd1a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:25.183040 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d13fd07e-55ea-4544-b51a-97ee6a10bd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:25.242055 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1357/2733] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:02:25.250601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b22c39-e416-484c-b638-bd61af795e8d None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:25.250896 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b22c39-e416-484c-b638-bd61af795e8d None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:25.251183 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b22c39-e416-484c-b638-bd61af795e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:25.270421 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b9b22c39-e416-484c-b638-bd61af795e8d None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:25.271652 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b9b22c39-e416-484c-b638-bd61af795e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:25.279846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1377/2734] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:25.284492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fd07e-55ea-4544-b51a-97ee6a10bd1a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:25.294592 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fd07e-55ea-4544-b51a-97ee6a10bd1a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:25.300603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdb52f-1e5a-472d-9ebf-db81d676e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:25.301349 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdb52f-1e5a-472d-9ebf-db81d676e418 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:25.301637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdb52f-1e5a-472d-9ebf-db81d676e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:25.312464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cdb52f-1e5a-472d-9ebf-db81d676e418 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:25.312464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdb52f-1e5a-472d-9ebf-db81d676e418 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:02:25.312464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdb52f-1e5a-472d-9ebf-db81d676e418 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:25.312464 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cdb52f-1e5a-472d-9ebf-db81d676e418 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:25.319685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1358/2735] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:02:25.324196 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b22c39-e416-484c-b638-bd61af795e8d None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:25.329737 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:25.335678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b22c39-e416-484c-b638-bd61af795e8d None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:25.340873 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18eacb-cb93-4f48-baa5-8cc83f147018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:25.346741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18eacb-cb93-4f48-baa5-8cc83f147018 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:25.346741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18eacb-cb93-4f48-baa5-8cc83f147018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:25.346741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18eacb-cb93-4f48-baa5-8cc83f147018 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:25.346741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18eacb-cb93-4f48-baa5-8cc83f147018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:25.346741 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18eacb-cb93-4f48-baa5-8cc83f147018 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:25.453684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1378/2736] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:25 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 17:02:25.457950 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdb52f-1e5a-472d-9ebf-db81d676e418 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:25.468112 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cdb52f-1e5a-472d-9ebf-db81d676e418 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:25.494993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:25.528105 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21c1fd-833b-4367-aa3a-6f974b47e249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:25.528775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21c1fd-833b-4367-aa3a-6f974b47e249 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:25.529090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21c1fd-833b-4367-aa3a-6f974b47e249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:25.529364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21c1fd-833b-4367-aa3a-6f974b47e249 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:25.530080 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21c1fd-833b-4367-aa3a-6f974b47e249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:25.532281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21c1fd-833b-4367-aa3a-6f974b47e249 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:25.623651 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1359/2737] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:25 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:25.628325 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18eacb-cb93-4f48-baa5-8cc83f147018 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:25.638895 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18eacb-cb93-4f48-baa5-8cc83f147018 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:25.643789 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f428a1-ae3f-4c82-9f71-6c113703c9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:25.644487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f428a1-ae3f-4c82-9f71-6c113703c9ac admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:25.644772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f428a1-ae3f-4c82-9f71-6c113703c9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:25.645086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f428a1-ae3f-4c82-9f71-6c113703c9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:25.645818 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f428a1-ae3f-4c82-9f71-6c113703c9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:25.647523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f428a1-ae3f-4c82-9f71-6c113703c9ac admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:25.658122 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1379/2738] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:25.662559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21c1fd-833b-4367-aa3a-6f974b47e249 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:25.676982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21c1fd-833b-4367-aa3a-6f974b47e249 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:25.682047 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248aa86f-b219-4fe4-8a8c-1d5df54c05b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:25.682706 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248aa86f-b219-4fe4-8a8c-1d5df54c05b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:25.682998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248aa86f-b219-4fe4-8a8c-1d5df54c05b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:25.683299 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248aa86f-b219-4fe4-8a8c-1d5df54c05b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:25.684039 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248aa86f-b219-4fe4-8a8c-1d5df54c05b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:25.685882 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248aa86f-b219-4fe4-8a8c-1d5df54c05b8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:25.697795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1360/2739] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:25.701323 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f428a1-ae3f-4c82-9f71-6c113703c9ac admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:25.711629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f428a1-ae3f-4c82-9f71-6c113703c9ac admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:25.718536 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8b724-44ce-4b66-be3d-3bc3b3750e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:25.719494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8b724-44ce-4b66-be3d-3bc3b3750e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:25.719784 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8b724-44ce-4b66-be3d-3bc3b3750e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:25.720059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8b724-44ce-4b66-be3d-3bc3b3750e2c admin admin] PATH_INFO: `/v3/projects/a5d59457835448be9303216a75357e0c/users/7df9358c25824d4e86393d9b412324c3/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:25.738376 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8b724-44ce-4b66-be3d-3bc3b3750e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5d59457835448be9303216a75357e0c, user_id=7df9358c25824d4e86393d9b412324c3, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:25.741230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8b724-44ce-4b66-be3d-3bc3b3750e2c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:25.811574 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1380/2740] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:25 2022] PUT /identity/v3/projects/a5d59457835448be9303216a75357e0c/users/7df9358c25824d4e86393d9b412324c3/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:25.822352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10944f3-6a65-4c38-abd4-4b7013a03a5b None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:25.822643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10944f3-6a65-4c38-abd4-4b7013a03a5b None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:25.822921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10944f3-6a65-4c38-abd4-4b7013a03a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:25.850427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e10944f3-6a65-4c38-abd4-4b7013a03a5b None None] MFA Rules not processed for user `7df9358c25824d4e86393d9b412324c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:25.851687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e10944f3-6a65-4c38-abd4-4b7013a03a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:25.916939 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1361/2741] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:25.922053 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8b724-44ce-4b66-be3d-3bc3b3750e2c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:25.988904 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8b724-44ce-4b66-be3d-3bc3b3750e2c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:25.993655 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6d835-5dd8-4388-b04e-f6c64abdc714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:25.994314 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6d835-5dd8-4388-b04e-f6c64abdc714 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:25.994608 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6d835-5dd8-4388-b04e-f6c64abdc714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:25.994884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6d835-5dd8-4388-b04e-f6c64abdc714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:25.995585 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6d835-5dd8-4388-b04e-f6c64abdc714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:25.997559 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6d835-5dd8-4388-b04e-f6c64abdc714 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:26.054467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1381/2742] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:25 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:26.058588 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10944f3-6a65-4c38-abd4-4b7013a03a5b None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:26.069818 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10944f3-6a65-4c38-abd4-4b7013a03a5b None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:26.128072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a372a-49c1-47f2-9102-a2e0bc2c5cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:26.128755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a372a-49c1-47f2-9102-a2e0bc2c5cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:26.129074 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a372a-49c1-47f2-9102-a2e0bc2c5cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:26.129356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a372a-49c1-47f2-9102-a2e0bc2c5cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:26.130053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a372a-49c1-47f2-9102-a2e0bc2c5cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:26.131996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a372a-49c1-47f2-9102-a2e0bc2c5cb6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:26.224653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1362/2743] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:26 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:26.229780 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6d835-5dd8-4388-b04e-f6c64abdc714 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:26.240745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6d835-5dd8-4388-b04e-f6c64abdc714 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:26.245769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7575a68-3fec-44e9-b32f-e8983b95e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:26.246336 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575a68-3fec-44e9-b32f-e8983b95e439 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:26.246679 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575a68-3fec-44e9-b32f-e8983b95e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:26.246985 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575a68-3fec-44e9-b32f-e8983b95e439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:26.247731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7575a68-3fec-44e9-b32f-e8983b95e439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:26.249389 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7575a68-3fec-44e9-b32f-e8983b95e439 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:26.260506 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1382/2744] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 17:02:26.265431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a372a-49c1-47f2-9102-a2e0bc2c5cb6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:26.275736 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a372a-49c1-47f2-9102-a2e0bc2c5cb6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:26.281559 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7908aa-7179-4d2a-8ea6-f55a411281eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:26.282475 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7908aa-7179-4d2a-8ea6-f55a411281eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:26.282762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7908aa-7179-4d2a-8ea6-f55a411281eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:26.283042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7908aa-7179-4d2a-8ea6-f55a411281eb admin admin] PATH_INFO: `/v3/projects/26f764fa8e3c4ac0b8d22341833ac4e6/users/277c9fa26918463583f8b251225673a1/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:26.298435 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7908aa-7179-4d2a-8ea6-f55a411281eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f764fa8e3c4ac0b8d22341833ac4e6, user_id=277c9fa26918463583f8b251225673a1, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:26.301590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7908aa-7179-4d2a-8ea6-f55a411281eb admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:26.384778 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:26.390356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1363/2745] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:26 2022] PUT /identity/v3/projects/26f764fa8e3c4ac0b8d22341833ac4e6/users/277c9fa26918463583f8b251225673a1/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:26.395361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7575a68-3fec-44e9-b32f-e8983b95e439 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:26.407355 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7575a68-3fec-44e9-b32f-e8983b95e439 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:26.473100 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d861b-5817-4a49-b3f3-48113ae85fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:26.473774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d861b-5817-4a49-b3f3-48113ae85fcf admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:26.474057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d861b-5817-4a49-b3f3-48113ae85fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:26.474327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d861b-5817-4a49-b3f3-48113ae85fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:26.475110 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d861b-5817-4a49-b3f3-48113ae85fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:26.476689 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d861b-5817-4a49-b3f3-48113ae85fcf admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:26.486920 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1383/2746] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:26.491273 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7908aa-7179-4d2a-8ea6-f55a411281eb admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:26.501795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7908aa-7179-4d2a-8ea6-f55a411281eb admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:26.506773 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3b455-d31f-4a75-b57e-f0adaa1b7899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:26.507687 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3b455-d31f-4a75-b57e-f0adaa1b7899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:26.507976 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3b455-d31f-4a75-b57e-f0adaa1b7899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:26.508255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3b455-d31f-4a75-b57e-f0adaa1b7899 admin admin] PATH_INFO: `/v3/projects/26f764fa8e3c4ac0b8d22341833ac4e6/users/277c9fa26918463583f8b251225673a1/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:26.510360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3b455-d31f-4a75-b57e-f0adaa1b7899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f764fa8e3c4ac0b8d22341833ac4e6, user_id=277c9fa26918463583f8b251225673a1, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:26.513227 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3b455-d31f-4a75-b57e-f0adaa1b7899 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:26.524440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1364/2747] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:26 2022] PUT /identity/v3/projects/26f764fa8e3c4ac0b8d22341833ac4e6/users/277c9fa26918463583f8b251225673a1/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:26.536446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381981b-e3a4-40b4-9572-2fc5ff9798de None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:26.536770 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381981b-e3a4-40b4-9572-2fc5ff9798de None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:26.537086 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381981b-e3a4-40b4-9572-2fc5ff9798de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:26.564463 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-2381981b-e3a4-40b4-9572-2fc5ff9798de None None] MFA Rules not processed for user `277c9fa26918463583f8b251225673a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:26.565998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2381981b-e3a4-40b4-9572-2fc5ff9798de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:26.570061 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:26.638986 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1384/2748] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:26.651520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3b455-d31f-4a75-b57e-f0adaa1b7899 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:26.653287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-90a3b455-d31f-4a75-b57e-f0adaa1b7899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:26.717494 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3b455-d31f-4a75-b57e-f0adaa1b7899 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:26.721744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5ebe2-f8b0-4852-86e0-2c9ffe3a2d4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:26.722550 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5ebe2-f8b0-4852-86e0-2c9ffe3a2d4c service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:26.722836 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5ebe2-f8b0-4852-86e0-2c9ffe3a2d4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:26.723116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5ebe2-f8b0-4852-86e0-2c9ffe3a2d4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:26.734598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5ebe2-f8b0-4852-86e0-2c9ffe3a2d4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:26.734869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5ebe2-f8b0-4852-86e0-2c9ffe3a2d4c service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:26.750885 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1365/2749] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:26.780262 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381981b-e3a4-40b4-9572-2fc5ff9798de None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:26.786426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-2381981b-e3a4-40b4-9572-2fc5ff9798de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:26.853129 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381981b-e3a4-40b4-9572-2fc5ff9798de None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:26.854489 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5ebe2-f8b0-4852-86e0-2c9ffe3a2d4c service cinder] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:26.855932 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-6ed5ebe2-f8b0-4852-86e0-2c9ffe3a2d4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:26.858094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:26.859004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:26.859285 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:26.859563 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] PATH_INFO: `/v3/users/bc3962d365a74ae68ff0b10b8830688b` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:26.860831 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc3962d365a74ae68ff0b10b8830688b)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:26.862766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:26.867829 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5ebe2-f8b0-4852-86e0-2c9ffe3a2d4c service cinder] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:26.871634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507efe07-8ee6-4b31-860d-b6cf19285d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:26.872295 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507efe07-8ee6-4b31-860d-b6cf19285d1f service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:26.872577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507efe07-8ee6-4b31-860d-b6cf19285d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:26.872862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507efe07-8ee6-4b31-860d-b6cf19285d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:26.931759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507efe07-8ee6-4b31-860d-b6cf19285d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:26.933356 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507efe07-8ee6-4b31-860d-b6cf19285d1f service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:26.943938 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1366/2750] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:27.037107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc3962d365a74ae68ff0b10b8830688b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:27.051470 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc3962d365a74ae68ff0b10b8830688b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:27.059573 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc3962d365a74ae68ff0b10b8830688b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:27.145083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc3962d365a74ae68ff0b10b8830688b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:27.146277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc3962d365a74ae68ff0b10b8830688b'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:27.154771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1385/2751] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:26 2022] DELETE /identity/v3/users/bc3962d365a74ae68ff0b10b8830688b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:27.160042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcba04-a057-41ab-a694-131be7e85605 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:27.160387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcba04-a057-41ab-a694-131be7e85605 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:27.160731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcba04-a057-41ab-a694-131be7e85605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:27.179398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-29dcba04-a057-41ab-a694-131be7e85605 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:27.180632 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-29dcba04-a057-41ab-a694-131be7e85605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:27.241778 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1367/2752] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:27.257936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:27.257936 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:27.277019 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:27.317702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7f939-fc75-46d9-aee2-d470b998c4d9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:27.321712 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f6b4d-0ae5-4c87-a151-5ac6f4ea5b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:27.322371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f6b4d-0ae5-4c87-a151-5ac6f4ea5b64 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:27.322659 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f6b4d-0ae5-4c87-a151-5ac6f4ea5b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:27.322940 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f6b4d-0ae5-4c87-a151-5ac6f4ea5b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:27.332615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f6b4d-0ae5-4c87-a151-5ac6f4ea5b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:27.334097 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f6b4d-0ae5-4c87-a151-5ac6f4ea5b64 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:27.346550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1386/2753] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:27.813762 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcba04-a057-41ab-a694-131be7e85605 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:27.815634 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-29dcba04-a057-41ab-a694-131be7e85605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:27.827879 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcba04-a057-41ab-a694-131be7e85605 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:27.832513 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:27.833813 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:27.834101 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:27.834383 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] PATH_INFO: `/v3/projects/46aa77f9a02243ca8585965d3d7863c1` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:27.835624 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46aa77f9a02243ca8585965d3d7863c1)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:27.837612 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:27.863698 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:27.945862 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46aa77f9a02243ca8585965d3d7863c1', 'request_id': 'req-1d5512bb-f8da-4e3c-9285-933326dfb873'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:27.952121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46aa77f9a02243ca8585965d3d7863c1', 'request_id': 'req-1d5512bb-f8da-4e3c-9285-933326dfb873'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:27.957802 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46aa77f9a02243ca8585965d3d7863c1', 'request_id': 'req-1d5512bb-f8da-4e3c-9285-933326dfb873'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:28.051409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] The token cache is being invalidate because project 46aa77f9a02243ca8585965d3d7863c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:28.051846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:28.052490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:28.054116 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1368/2754] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:27 2022] DELETE /identity/v3/projects/46aa77f9a02243ca8585965d3d7863c1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:02:28.058112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f6b4d-0ae5-4c87-a151-5ac6f4ea5b64 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:28.069503 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f6b4d-0ae5-4c87-a151-5ac6f4ea5b64 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:28.126697 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:28.127493 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:28.127777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:28.128103 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] PATH_INFO: `/v3/users/02a87ca3f9e94b98bd0038c977d2d7fb` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:28.129287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a87ca3f9e94b98bd0038c977d2d7fb)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:28.131220 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:28.265100 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a87ca3f9e94b98bd0038c977d2d7fb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:28.277001 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a87ca3f9e94b98bd0038c977d2d7fb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:28.285310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a87ca3f9e94b98bd0038c977d2d7fb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:28.342507 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a87ca3f9e94b98bd0038c977d2d7fb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:28.343647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a87ca3f9e94b98bd0038c977d2d7fb'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:28.351303 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1387/2755] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:28 2022] DELETE /identity/v3/users/02a87ca3f9e94b98bd0038c977d2d7fb => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:28.801979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:28.803468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:28.816792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5512bb-f8da-4e3c-9285-933326dfb873 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:28.875190 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:28.876053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:28.876333 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:28.876608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] PATH_INFO: `/v3/projects/ba5f34946323420f9819d5f51cb41a57` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:28.877825 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba5f34946323420f9819d5f51cb41a57)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:28.879784 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:29.034701 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba5f34946323420f9819d5f51cb41a57', 'request_id': 'req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:29.040982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba5f34946323420f9819d5f51cb41a57', 'request_id': 'req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:29.046839 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba5f34946323420f9819d5f51cb41a57', 'request_id': 'req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:29.137182 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] The token cache is being invalidate because project ba5f34946323420f9819d5f51cb41a57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:29.139779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:29.139779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:29.139779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1369/2756] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:28 2022] DELETE /identity/v3/projects/ba5f34946323420f9819d5f51cb41a57 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:02:29.144361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:29.145990 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:29.163650 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129547a-176c-4e18-a39f-09ba17031a28 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:29.209396 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:29.227553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cabda-cab1-4345-9eb1-df3cc2ea378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:29.228430 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cabda-cab1-4345-9eb1-df3cc2ea378c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:29.228833 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cabda-cab1-4345-9eb1-df3cc2ea378c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:29.229150 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cabda-cab1-4345-9eb1-df3cc2ea378c admin admin] PATH_INFO: `/v3/users/cf90906a38024d05aeabb03d29079c90` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:29.230340 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cabda-cab1-4345-9eb1-df3cc2ea378c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf90906a38024d05aeabb03d29079c90)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:29.232351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cabda-cab1-4345-9eb1-df3cc2ea378c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:29.478284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-692cabda-cab1-4345-9eb1-df3cc2ea378c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf90906a38024d05aeabb03d29079c90'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:29.490569 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-692cabda-cab1-4345-9eb1-df3cc2ea378c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf90906a38024d05aeabb03d29079c90'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:29.498915 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-692cabda-cab1-4345-9eb1-df3cc2ea378c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf90906a38024d05aeabb03d29079c90'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:29.577115 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-692cabda-cab1-4345-9eb1-df3cc2ea378c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf90906a38024d05aeabb03d29079c90'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:29.577756 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-692cabda-cab1-4345-9eb1-df3cc2ea378c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf90906a38024d05aeabb03d29079c90'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:29.584717 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1388/2757] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:29 2022] DELETE /identity/v3/users/cf90906a38024d05aeabb03d29079c90 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:30.019889 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:30.025699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:30.036515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc61c52-dc8b-4edc-8226-fc1e43c4531e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:30.080512 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:30.096756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:30.097662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:30.097965 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:30.098229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] PATH_INFO: `/v3/projects/46217432938943328ae4a9b2c053340b` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:30.099431 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46217432938943328ae4a9b2c053340b)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:30.101562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:30.207755 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46217432938943328ae4a9b2c053340b', 'request_id': 'req-64142328-2950-4e28-8f0e-5bd8c373b7f8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:30.213933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46217432938943328ae4a9b2c053340b', 'request_id': 'req-64142328-2950-4e28-8f0e-5bd8c373b7f8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:30.219682 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46217432938943328ae4a9b2c053340b', 'request_id': 'req-64142328-2950-4e28-8f0e-5bd8c373b7f8'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:30.302493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] The token cache is being invalidate because project 46217432938943328ae4a9b2c053340b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:30.302712 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:30.303522 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-64142328-2950-4e28-8f0e-5bd8c373b7f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:30.305276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1370/2758] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:30 2022] DELETE /identity/v3/projects/46217432938943328ae4a9b2c053340b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:30.311367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabb138-5174-4f6b-b5bf-ff939eb96936 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:30.311972 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabb138-5174-4f6b-b5bf-ff939eb96936 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:30.311972 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabb138-5174-4f6b-b5bf-ff939eb96936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:30.332940 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3cabb138-5174-4f6b-b5bf-ff939eb96936 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:30.334154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3cabb138-5174-4f6b-b5bf-ff939eb96936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:30.394327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1389/2759] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:30.403069 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1ae7b-781e-4453-bff9-2c6ac88da81a None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:30.403361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1ae7b-781e-4453-bff9-2c6ac88da81a None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:30.403671 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1ae7b-781e-4453-bff9-2c6ac88da81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:30.424239 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-16c1ae7b-781e-4453-bff9-2c6ac88da81a None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:30.425492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-16c1ae7b-781e-4453-bff9-2c6ac88da81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:30.433934 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1371/2760] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:30.438306 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabb138-5174-4f6b-b5bf-ff939eb96936 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:30.448961 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabb138-5174-4f6b-b5bf-ff939eb96936 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:30.453711 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce93cca-d47c-41c0-bc25-aad17ecc7a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:30.454479 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce93cca-d47c-41c0-bc25-aad17ecc7a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:30.454763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce93cca-d47c-41c0-bc25-aad17ecc7a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:30.455040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce93cca-d47c-41c0-bc25-aad17ecc7a7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:30.455765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce93cca-d47c-41c0-bc25-aad17ecc7a7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:02:30.456070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce93cca-d47c-41c0-bc25-aad17ecc7a7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:30.457700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce93cca-d47c-41c0-bc25-aad17ecc7a7d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:30.469604 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1390/2761] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:02:30.474441 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1ae7b-781e-4453-bff9-2c6ac88da81a None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:30.484406 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1ae7b-781e-4453-bff9-2c6ac88da81a None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:30.489787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880c5f3-cff1-4114-b80d-265571a7218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:30.490421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880c5f3-cff1-4114-b80d-265571a7218e admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:30.490704 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880c5f3-cff1-4114-b80d-265571a7218e admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:30.490979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880c5f3-cff1-4114-b80d-265571a7218e admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:30.491674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880c5f3-cff1-4114-b80d-265571a7218e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:30.493630 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880c5f3-cff1-4114-b80d-265571a7218e admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:30.635963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1372/2762] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:30 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:30.640310 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce93cca-d47c-41c0-bc25-aad17ecc7a7d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:30.650801 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce93cca-d47c-41c0-bc25-aad17ecc7a7d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:30.698647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:30.710451 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30149916-000e-4a7c-89c6-e3c23f9497ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:30.711130 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30149916-000e-4a7c-89c6-e3c23f9497ad admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:30.711417 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30149916-000e-4a7c-89c6-e3c23f9497ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:30.711698 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30149916-000e-4a7c-89c6-e3c23f9497ad admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:30.712411 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30149916-000e-4a7c-89c6-e3c23f9497ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:30.714342 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30149916-000e-4a7c-89c6-e3c23f9497ad admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:30.822868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1391/2763] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:30 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:30.827367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880c5f3-cff1-4114-b80d-265571a7218e admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:30.838628 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880c5f3-cff1-4114-b80d-265571a7218e admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:30.844702 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377923b-0a71-4550-aeda-141d9efe9f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:30.845405 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377923b-0a71-4550-aeda-141d9efe9f85 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:30.845686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377923b-0a71-4550-aeda-141d9efe9f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:30.845962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377923b-0a71-4550-aeda-141d9efe9f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:30.846692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377923b-0a71-4550-aeda-141d9efe9f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:30.848282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377923b-0a71-4550-aeda-141d9efe9f85 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:30.860038 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1373/2764] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:30.864189 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30149916-000e-4a7c-89c6-e3c23f9497ad admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:30.874347 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30149916-000e-4a7c-89c6-e3c23f9497ad admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:30.880359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc5ab8-340b-47f1-910d-9588bf871ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:30.883408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc5ab8-340b-47f1-910d-9588bf871ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:30.883408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc5ab8-340b-47f1-910d-9588bf871ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:30.883408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc5ab8-340b-47f1-910d-9588bf871ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:30.883408 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc5ab8-340b-47f1-910d-9588bf871ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:30.884158 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc5ab8-340b-47f1-910d-9588bf871ee5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:30.900014 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1392/2765] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 17:02:30.900146 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377923b-0a71-4550-aeda-141d9efe9f85 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:30.910817 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377923b-0a71-4550-aeda-141d9efe9f85 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:30.917228 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d64de-f7af-402e-837a-af7b0a3ccba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:30.918183 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d64de-f7af-402e-837a-af7b0a3ccba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:30.918439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d64de-f7af-402e-837a-af7b0a3ccba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:30.918717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d64de-f7af-402e-837a-af7b0a3ccba2 admin admin] PATH_INFO: `/v3/projects/31bf6345dcd5437fabe949c7c9c24b0c/users/5d397c701e8446d19c84c3f4991b9771/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:30.941229 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d64de-f7af-402e-837a-af7b0a3ccba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31bf6345dcd5437fabe949c7c9c24b0c, user_id=5d397c701e8446d19c84c3f4991b9771, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:30.944451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d64de-f7af-402e-837a-af7b0a3ccba2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:31.040409 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1374/2766] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:30 2022] PUT /identity/v3/projects/31bf6345dcd5437fabe949c7c9c24b0c/users/5d397c701e8446d19c84c3f4991b9771/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:31.105015 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ef5aa-d3db-4262-978a-93928128d709 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:31.105395 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ef5aa-d3db-4262-978a-93928128d709 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:31.105592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ef5aa-d3db-4262-978a-93928128d709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:31.133212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d15ef5aa-d3db-4262-978a-93928128d709 None None] MFA Rules not processed for user `5d397c701e8446d19c84c3f4991b9771`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:31.134458 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d15ef5aa-d3db-4262-978a-93928128d709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:31.202707 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1393/2767] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:31.207692 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d64de-f7af-402e-837a-af7b0a3ccba2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:31.272020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d64de-f7af-402e-837a-af7b0a3ccba2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:31.276819 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4058e-76d1-4bea-91ee-4b46429f62e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:31.277504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4058e-76d1-4bea-91ee-4b46429f62e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:31.277795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4058e-76d1-4bea-91ee-4b46429f62e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:31.278073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4058e-76d1-4bea-91ee-4b46429f62e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:31.278832 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4058e-76d1-4bea-91ee-4b46429f62e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:31.280738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4058e-76d1-4bea-91ee-4b46429f62e6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:31.294929 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:31.371461 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1375/2768] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:31 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 17:02:31.372528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ef5aa-d3db-4262-978a-93928128d709 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:31.383599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ef5aa-d3db-4262-978a-93928128d709 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:31.445092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345c3cf-665f-42fb-bfc3-459a33fb9d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:31.445826 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345c3cf-665f-42fb-bfc3-459a33fb9d35 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:31.446171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345c3cf-665f-42fb-bfc3-459a33fb9d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:31.446489 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f345c3cf-665f-42fb-bfc3-459a33fb9d35 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:31.447250 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345c3cf-665f-42fb-bfc3-459a33fb9d35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:31.449276 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f345c3cf-665f-42fb-bfc3-459a33fb9d35 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:31.561590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1394/2769] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:31 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:31.569271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4058e-76d1-4bea-91ee-4b46429f62e6 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:31.580387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4058e-76d1-4bea-91ee-4b46429f62e6 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:31.585147 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a2a8c-8154-4d87-b706-1a5bdddcd6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:31.585808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a2a8c-8154-4d87-b706-1a5bdddcd6ba admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:31.586089 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a2a8c-8154-4d87-b706-1a5bdddcd6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:31.586367 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a2a8c-8154-4d87-b706-1a5bdddcd6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:31.587176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a2a8c-8154-4d87-b706-1a5bdddcd6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:31.588808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a2a8c-8154-4d87-b706-1a5bdddcd6ba admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:31.599885 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1376/2770] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:31.604227 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345c3cf-665f-42fb-bfc3-459a33fb9d35 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:31.614681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f345c3cf-665f-42fb-bfc3-459a33fb9d35 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:31.619319 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b6537-d553-4185-b48e-3a98fd149cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:31.620188 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b6537-d553-4185-b48e-3a98fd149cee admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:31.620468 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b6537-d553-4185-b48e-3a98fd149cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:31.620740 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b6537-d553-4185-b48e-3a98fd149cee admin admin] PATH_INFO: `/v3/projects/4b69745eb9164f8da36d9489162207d7/users/283b66e1059e4c85a67d6c1a1e24f35c/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:31.636405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b6537-d553-4185-b48e-3a98fd149cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b69745eb9164f8da36d9489162207d7, user_id=283b66e1059e4c85a67d6c1a1e24f35c, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:31.639371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b6537-d553-4185-b48e-3a98fd149cee admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:31.726070 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1395/2771] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:31 2022] PUT /identity/v3/projects/4b69745eb9164f8da36d9489162207d7/users/283b66e1059e4c85a67d6c1a1e24f35c/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:31.730345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a2a8c-8154-4d87-b706-1a5bdddcd6ba admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:31.741797 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a2a8c-8154-4d87-b706-1a5bdddcd6ba admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:31.798831 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66c451-cb82-40f6-b33f-a3acd19c3b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:31.799501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66c451-cb82-40f6-b33f-a3acd19c3b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:31.799794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66c451-cb82-40f6-b33f-a3acd19c3b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:31.800072 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66c451-cb82-40f6-b33f-a3acd19c3b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:31.800808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66c451-cb82-40f6-b33f-a3acd19c3b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:31.802444 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66c451-cb82-40f6-b33f-a3acd19c3b87 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:31.812573 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1377/2772] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:31.817772 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b6537-d553-4185-b48e-3a98fd149cee admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:31.829011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b6537-d553-4185-b48e-3a98fd149cee admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:31.834362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b31c8-b723-48a4-8191-a20b30006f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:31.835275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b31c8-b723-48a4-8191-a20b30006f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:31.835560 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b31c8-b723-48a4-8191-a20b30006f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:31.835836 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b31c8-b723-48a4-8191-a20b30006f8d admin admin] PATH_INFO: `/v3/projects/4b69745eb9164f8da36d9489162207d7/users/283b66e1059e4c85a67d6c1a1e24f35c/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:31.840644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b31c8-b723-48a4-8191-a20b30006f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b69745eb9164f8da36d9489162207d7, user_id=283b66e1059e4c85a67d6c1a1e24f35c, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:31.840644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b31c8-b723-48a4-8191-a20b30006f8d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:31.852594 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1396/2773] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:31 2022] PUT /identity/v3/projects/4b69745eb9164f8da36d9489162207d7/users/283b66e1059e4c85a67d6c1a1e24f35c/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:31.863719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27388176-4f2b-4de1-bd28-a148b54c7a93 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:31.864015 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27388176-4f2b-4de1-bd28-a148b54c7a93 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:31.864298 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27388176-4f2b-4de1-bd28-a148b54c7a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:31.892681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-27388176-4f2b-4de1-bd28-a148b54c7a93 None None] MFA Rules not processed for user `283b66e1059e4c85a67d6c1a1e24f35c`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:31.893942 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-27388176-4f2b-4de1-bd28-a148b54c7a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:31.969263 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b31c8-b723-48a4-8191-a20b30006f8d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:31.971326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fc0b31c8-b723-48a4-8191-a20b30006f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:31.975643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1378/2774] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:02:31.987825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:31.987962 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27388176-4f2b-4de1-bd28-a148b54c7a93 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:31.989075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-27388176-4f2b-4de1-bd28-a148b54c7a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:32.037478 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b31c8-b723-48a4-8191-a20b30006f8d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:32.042236 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:32.043106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:32.043419 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:32.043727 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] PATH_INFO: `/v3/users/7df9358c25824d4e86393d9b412324c3` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:32.045038 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df9358c25824d4e86393d9b412324c3)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:32.046520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27388176-4f2b-4de1-bd28-a148b54c7a93 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:32.047032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:32.050871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1829f-e465-47a0-af0d-ce1d85fc6e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:32.051517 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1829f-e465-47a0-af0d-ce1d85fc6e85 service cinder] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:32.051804 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1829f-e465-47a0-af0d-ce1d85fc6e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:32.052080 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1829f-e465-47a0-af0d-ce1d85fc6e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:32.070429 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:32.109657 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1829f-e465-47a0-af0d-ce1d85fc6e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:32.111747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1829f-e465-47a0-af0d-ce1d85fc6e85 service cinder] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:32.121376 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1379/2775] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:02:32.215267 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df9358c25824d4e86393d9b412324c3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:32.230777 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df9358c25824d4e86393d9b412324c3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:32.240058 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df9358c25824d4e86393d9b412324c3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:32.297691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df9358c25824d4e86393d9b412324c3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:32.298932 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df9358c25824d4e86393d9b412324c3'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:32.307703 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1397/2776] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:31 2022] DELETE /identity/v3/users/7df9358c25824d4e86393d9b412324c3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:02:32.312219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0427a2-83ed-407e-843c-655d36d0a8be None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:32.312518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0427a2-83ed-407e-843c-655d36d0a8be None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:32.312823 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0427a2-83ed-407e-843c-655d36d0a8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:32.330747 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-7f0427a2-83ed-407e-843c-655d36d0a8be None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:32.331987 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7f0427a2-83ed-407e-843c-655d36d0a8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:32.393163 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1380/2777] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:32.403320 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:32.404758 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:32.464820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97711a-a45b-4124-a5ad-047b4baa86ab admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:32.468613 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c64eb-c815-4820-988c-cde44f41941b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:32.473425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c64eb-c815-4820-988c-cde44f41941b service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:32.473425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c64eb-c815-4820-988c-cde44f41941b service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:32.473425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c64eb-c815-4820-988c-cde44f41941b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:32.479162 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c64eb-c815-4820-988c-cde44f41941b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:32.480602 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c64eb-c815-4820-988c-cde44f41941b service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:32.490181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1398/2778] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:32.952969 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0427a2-83ed-407e-843c-655d36d0a8be None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:32.954575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-7f0427a2-83ed-407e-843c-655d36d0a8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:32.967374 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0427a2-83ed-407e-843c-655d36d0a8be None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:32.973943 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:32.974997 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:32.975292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:32.975582 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] PATH_INFO: `/v3/projects/a5d59457835448be9303216a75357e0c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:32.976864 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5d59457835448be9303216a75357e0c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:32.986233 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:33.121185 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5d59457835448be9303216a75357e0c', 'request_id': 'req-4ec48101-39c3-44f3-9feb-111b576ac298'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:33.127424 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5d59457835448be9303216a75357e0c', 'request_id': 'req-4ec48101-39c3-44f3-9feb-111b576ac298'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:33.133271 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5d59457835448be9303216a75357e0c', 'request_id': 'req-4ec48101-39c3-44f3-9feb-111b576ac298'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:33.217035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] The token cache is being invalidate because project a5d59457835448be9303216a75357e0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:33.217323 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:33.217953 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:33.219504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1381/2779] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:32 2022] DELETE /identity/v3/projects/a5d59457835448be9303216a75357e0c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:33.224136 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c64eb-c815-4820-988c-cde44f41941b service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:33.234941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c64eb-c815-4820-988c-cde44f41941b service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:33.265003 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:33.293012 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c74a5-7b33-4761-af8a-eed9b16a0d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:33.293895 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c74a5-7b33-4761-af8a-eed9b16a0d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:33.294181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c74a5-7b33-4761-af8a-eed9b16a0d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:33.294455 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c74a5-7b33-4761-af8a-eed9b16a0d4b admin admin] PATH_INFO: `/v3/users/277c9fa26918463583f8b251225673a1` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:33.295660 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c74a5-7b33-4761-af8a-eed9b16a0d4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=277c9fa26918463583f8b251225673a1)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:33.297629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c74a5-7b33-4761-af8a-eed9b16a0d4b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:33.494111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ff7c74a5-7b33-4761-af8a-eed9b16a0d4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '277c9fa26918463583f8b251225673a1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:33.506510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ff7c74a5-7b33-4761-af8a-eed9b16a0d4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '277c9fa26918463583f8b251225673a1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:33.513586 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ff7c74a5-7b33-4761-af8a-eed9b16a0d4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '277c9fa26918463583f8b251225673a1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:33.571179 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ff7c74a5-7b33-4761-af8a-eed9b16a0d4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '277c9fa26918463583f8b251225673a1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:33.571884 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-ff7c74a5-7b33-4761-af8a-eed9b16a0d4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '277c9fa26918463583f8b251225673a1'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:33.578934 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1399/2780] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:33 2022] DELETE /identity/v3/users/277c9fa26918463583f8b251225673a1 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:33.967373 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:33.969000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:33.983705 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec48101-39c3-44f3-9feb-111b576ac298 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:34.029488 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:34.045776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:34.046794 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:34.047084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:34.047362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] PATH_INFO: `/v3/projects/26f764fa8e3c4ac0b8d22341833ac4e6` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:34.048586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26f764fa8e3c4ac0b8d22341833ac4e6)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:34.050648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:34.148738 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26f764fa8e3c4ac0b8d22341833ac4e6', 'request_id': 'req-fda11b44-d779-47c5-9593-6d347a9c8513'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:34.154956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26f764fa8e3c4ac0b8d22341833ac4e6', 'request_id': 'req-fda11b44-d779-47c5-9593-6d347a9c8513'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:34.160659 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26f764fa8e3c4ac0b8d22341833ac4e6', 'request_id': 'req-fda11b44-d779-47c5-9593-6d347a9c8513'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:34.233574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] The token cache is being invalidate because project 26f764fa8e3c4ac0b8d22341833ac4e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:34.233871 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:34.234541 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-fda11b44-d779-47c5-9593-6d347a9c8513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:34.236679 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1382/2781] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:33 2022] DELETE /identity/v3/projects/26f764fa8e3c4ac0b8d22341833ac4e6 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:34.241626 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd640a-0521-46dc-92bb-a48d0d7fd5d7 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:34.241918 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd640a-0521-46dc-92bb-a48d0d7fd5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:34.242205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd640a-0521-46dc-92bb-a48d0d7fd5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:34.261161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-aadd640a-0521-46dc-92bb-a48d0d7fd5d7 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:34.262438 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-aadd640a-0521-46dc-92bb-a48d0d7fd5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:34.323619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1400/2782] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:34.331439 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0476ccb-8e47-49fa-b46f-e025552c37db None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:34.331719 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0476ccb-8e47-49fa-b46f-e025552c37db None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:34.331998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0476ccb-8e47-49fa-b46f-e025552c37db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:34.350654 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d0476ccb-8e47-49fa-b46f-e025552c37db None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:34.351857 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d0476ccb-8e47-49fa-b46f-e025552c37db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:34.359830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1383/2783] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:34.364683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd640a-0521-46dc-92bb-a48d0d7fd5d7 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:34.374824 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd640a-0521-46dc-92bb-a48d0d7fd5d7 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:34.379958 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c401c-0a26-4d79-8ed0-1bfd15598073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:34.380623 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c401c-0a26-4d79-8ed0-1bfd15598073 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:34.380967 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c401c-0a26-4d79-8ed0-1bfd15598073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:34.381248 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c401c-0a26-4d79-8ed0-1bfd15598073 admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:34.381971 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c401c-0a26-4d79-8ed0-1bfd15598073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:02:34.382284 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c401c-0a26-4d79-8ed0-1bfd15598073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:34.384017 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5c401c-0a26-4d79-8ed0-1bfd15598073 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:34.395931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1401/2784] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:02:34.400283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0476ccb-8e47-49fa-b46f-e025552c37db None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:34.409911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0476ccb-8e47-49fa-b46f-e025552c37db None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:34.414690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca762a7c-c22f-4691-bf42-adc21595cedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:34.415353 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca762a7c-c22f-4691-bf42-adc21595cedc admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:34.415637 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca762a7c-c22f-4691-bf42-adc21595cedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:34.415911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca762a7c-c22f-4691-bf42-adc21595cedc admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:34.416601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca762a7c-c22f-4691-bf42-adc21595cedc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:34.418553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca762a7c-c22f-4691-bf42-adc21595cedc admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:34.505870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1384/2785] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:34 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:34.510049 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c401c-0a26-4d79-8ed0-1bfd15598073 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:34.520394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c401c-0a26-4d79-8ed0-1bfd15598073 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:34.556120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:34.581431 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b2f1-9005-4b44-88e8-2bc899f51a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:34.582082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b2f1-9005-4b44-88e8-2bc899f51a53 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:34.582377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b2f1-9005-4b44-88e8-2bc899f51a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:34.582657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b2f1-9005-4b44-88e8-2bc899f51a53 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:34.583486 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0b2f1-9005-4b44-88e8-2bc899f51a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:34.585446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0b2f1-9005-4b44-88e8-2bc899f51a53 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:34.642603 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1402/2786] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:34 2022] POST /identity/v3/users => generated 290 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:34.646717 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca762a7c-c22f-4691-bf42-adc21595cedc admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:34.657125 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca762a7c-c22f-4691-bf42-adc21595cedc admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:34.661982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb558afb-8cd5-477c-9ff1-4a8bee270004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:34.664904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb558afb-8cd5-477c-9ff1-4a8bee270004 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:34.664904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb558afb-8cd5-477c-9ff1-4a8bee270004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:34.664904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb558afb-8cd5-477c-9ff1-4a8bee270004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:34.664904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb558afb-8cd5-477c-9ff1-4a8bee270004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:34.665656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb558afb-8cd5-477c-9ff1-4a8bee270004 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:34.676248 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1385/2787] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:34.680265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b2f1-9005-4b44-88e8-2bc899f51a53 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:34.691141 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b2f1-9005-4b44-88e8-2bc899f51a53 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:34.695810 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9dcefa-bc23-4999-ad13-9f0ed2d0141d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:34.696456 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dcefa-bc23-4999-ad13-9f0ed2d0141d admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:34.696738 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dcefa-bc23-4999-ad13-9f0ed2d0141d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:34.697050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dcefa-bc23-4999-ad13-9f0ed2d0141d admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:34.697771 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9dcefa-bc23-4999-ad13-9f0ed2d0141d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:34.699349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9dcefa-bc23-4999-ad13-9f0ed2d0141d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:34.711874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1403/2788] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:34.716065 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb558afb-8cd5-477c-9ff1-4a8bee270004 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:34.726288 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb558afb-8cd5-477c-9ff1-4a8bee270004 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:34.731027 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58ced8-15c3-4080-a40f-881b10f937fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:34.731905 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58ced8-15c3-4080-a40f-881b10f937fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:34.732184 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58ced8-15c3-4080-a40f-881b10f937fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:34.732457 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58ced8-15c3-4080-a40f-881b10f937fe admin admin] PATH_INFO: `/v3/projects/e0316cef126347c1bc5b46bba46ddb06/users/cb331830c4f847c4ade846d0e2698e1c/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:34.748144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58ced8-15c3-4080-a40f-881b10f937fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0316cef126347c1bc5b46bba46ddb06, user_id=cb331830c4f847c4ade846d0e2698e1c, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:34.750777 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58ced8-15c3-4080-a40f-881b10f937fe admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:34.852161 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1386/2789] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:34 2022] PUT /identity/v3/projects/e0316cef126347c1bc5b46bba46ddb06/users/cb331830c4f847c4ade846d0e2698e1c/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:34.862870 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916ca27-a37c-4430-92d7-271d0bf72b4e None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:34.863193 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916ca27-a37c-4430-92d7-271d0bf72b4e None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:34.863473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916ca27-a37c-4430-92d7-271d0bf72b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:34.892384 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-d916ca27-a37c-4430-92d7-271d0bf72b4e None None] MFA Rules not processed for user `cb331830c4f847c4ade846d0e2698e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:34.893649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d916ca27-a37c-4430-92d7-271d0bf72b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:34.959808 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1404/2790] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:34.964258 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58ced8-15c3-4080-a40f-881b10f937fe admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:35.026989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58ced8-15c3-4080-a40f-881b10f937fe admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:35.031739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a7624-3b44-4c61-a72f-aec5f50b70c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:35.032392 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a7624-3b44-4c61-a72f-aec5f50b70c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:35.032676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a7624-3b44-4c61-a72f-aec5f50b70c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:35.032984 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a7624-3b44-4c61-a72f-aec5f50b70c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:35.033681 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a7624-3b44-4c61-a72f-aec5f50b70c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:35.035579 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a7624-3b44-4c61-a72f-aec5f50b70c8 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:35.049428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:35.109909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1387/2791] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:34 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:35.114515 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916ca27-a37c-4430-92d7-271d0bf72b4e None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:35.125404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916ca27-a37c-4430-92d7-271d0bf72b4e None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:35.182973 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3bb6b-0111-4df1-ba6f-c3fe880ba7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:35.183751 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3bb6b-0111-4df1-ba6f-c3fe880ba7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:35.184040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3bb6b-0111-4df1-ba6f-c3fe880ba7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:35.184318 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3bb6b-0111-4df1-ba6f-c3fe880ba7e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:35.185056 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3bb6b-0111-4df1-ba6f-c3fe880ba7e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:35.187002 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3bb6b-0111-4df1-ba6f-c3fe880ba7e8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:35.253669 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a7624-3b44-4c61-a72f-aec5f50b70c8 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:35.255539 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-bb5a7624-3b44-4c61-a72f-aec5f50b70c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:35.268245 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a7624-3b44-4c61-a72f-aec5f50b70c8 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:35.273473 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eef9f6-bb41-44e3-adc6-b5f90680edae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:35.274414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eef9f6-bb41-44e3-adc6-b5f90680edae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:35.279371 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eef9f6-bb41-44e3-adc6-b5f90680edae admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:35.279481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1405/2792] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:35 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:35.279580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eef9f6-bb41-44e3-adc6-b5f90680edae admin admin] PATH_INFO: `/v3/users/5d397c701e8446d19c84c3f4991b9771` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:35.279580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eef9f6-bb41-44e3-adc6-b5f90680edae admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d397c701e8446d19c84c3f4991b9771)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:35.279580 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eef9f6-bb41-44e3-adc6-b5f90680edae admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:35.282087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3bb6b-0111-4df1-ba6f-c3fe880ba7e8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:35.293476 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3bb6b-0111-4df1-ba6f-c3fe880ba7e8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:35.298995 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d3f9a-609c-4a7a-b97f-aeb5a83d8cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:35.299664 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d3f9a-609c-4a7a-b97f-aeb5a83d8cef admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:35.299948 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d3f9a-609c-4a7a-b97f-aeb5a83d8cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:35.300223 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d3f9a-609c-4a7a-b97f-aeb5a83d8cef admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:35.301136 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d3f9a-609c-4a7a-b97f-aeb5a83d8cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:35.302620 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d3f9a-609c-4a7a-b97f-aeb5a83d8cef admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:35.314246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1406/2793] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 17:02:35.318685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d3f9a-609c-4a7a-b97f-aeb5a83d8cef admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:35.328446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d3f9a-609c-4a7a-b97f-aeb5a83d8cef admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:35.333706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621f86-4503-4740-a50c-3bab338a22b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:35.334022 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621f86-4503-4740-a50c-3bab338a22b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:35.334300 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621f86-4503-4740-a50c-3bab338a22b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:35.334580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621f86-4503-4740-a50c-3bab338a22b1 admin admin] PATH_INFO: `/v3/projects/6846e3d3c19a4b469c4667802e951067/users/15b9a63f2a1244eb94518db1838bd212/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:35.350211 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8621f86-4503-4740-a50c-3bab338a22b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6846e3d3c19a4b469c4667802e951067, user_id=15b9a63f2a1244eb94518db1838bd212, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:35.352921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8621f86-4503-4740-a50c-3bab338a22b1 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:35.442567 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:35.447745 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1407/2794] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:35 2022] PUT /identity/v3/projects/6846e3d3c19a4b469c4667802e951067/users/15b9a63f2a1244eb94518db1838bd212/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:02:35.451557 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621f86-4503-4740-a50c-3bab338a22b1 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:35.499781 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-46eef9f6-bb41-44e3-adc6-b5f90680edae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d397c701e8446d19c84c3f4991b9771'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:35.507518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-46eef9f6-bb41-44e3-adc6-b5f90680edae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d397c701e8446d19c84c3f4991b9771'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:35.513921 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621f86-4503-4740-a50c-3bab338a22b1 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:35.520259 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7a08d-e7e9-4662-a683-7d6c3fc4e5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:35.520676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-46eef9f6-bb41-44e3-adc6-b5f90680edae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d397c701e8446d19c84c3f4991b9771'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:35.521011 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7a08d-e7e9-4662-a683-7d6c3fc4e5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:35.521305 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7a08d-e7e9-4662-a683-7d6c3fc4e5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:35.521587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7a08d-e7e9-4662-a683-7d6c3fc4e5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:35.522330 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7a08d-e7e9-4662-a683-7d6c3fc4e5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:35.525868 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7a08d-e7e9-4662-a683-7d6c3fc4e5b7 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:35.534527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1408/2795] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 17:02:35.538917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7a08d-e7e9-4662-a683-7d6c3fc4e5b7 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:35.552335 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7a08d-e7e9-4662-a683-7d6c3fc4e5b7 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:35.558215 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a31008-994a-483e-8c9e-fc92c01bd765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:35.558974 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a31008-994a-483e-8c9e-fc92c01bd765 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:35.559265 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a31008-994a-483e-8c9e-fc92c01bd765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:35.559544 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a31008-994a-483e-8c9e-fc92c01bd765 admin admin] PATH_INFO: `/v3/projects/6846e3d3c19a4b469c4667802e951067/users/15b9a63f2a1244eb94518db1838bd212/roles/81457691f3a2456fa33358cb6a922d05` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:35.561615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a31008-994a-483e-8c9e-fc92c01bd765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6846e3d3c19a4b469c4667802e951067, user_id=15b9a63f2a1244eb94518db1838bd212, role_id=81457691f3a2456fa33358cb6a922d05)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:35.566866 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a31008-994a-483e-8c9e-fc92c01bd765 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:35.581345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1409/2796] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:35 2022] PUT /identity/v3/projects/6846e3d3c19a4b469c4667802e951067/users/15b9a63f2a1244eb94518db1838bd212/roles/81457691f3a2456fa33358cb6a922d05 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:02:35.593327 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd436802-5242-49fd-abff-0a81c54d0cb9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:35.593774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd436802-5242-49fd-abff-0a81c54d0cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:35.594238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd436802-5242-49fd-abff-0a81c54d0cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:35.627984 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-dd436802-5242-49fd-abff-0a81c54d0cb9 None None] MFA Rules not processed for user `15b9a63f2a1244eb94518db1838bd212`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:35.629258 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dd436802-5242-49fd-abff-0a81c54d0cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:35.640419 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-46eef9f6-bb41-44e3-adc6-b5f90680edae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d397c701e8446d19c84c3f4991b9771'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:35.641171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-46eef9f6-bb41-44e3-adc6-b5f90680edae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d397c701e8446d19c84c3f4991b9771'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:35.649526 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1388/2797] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:35 2022] DELETE /identity/v3/users/5d397c701e8446d19c84c3f4991b9771 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:02:35.653892 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a779bd-c330-43c3-8bbb-255e8e77a469 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:35.654178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a779bd-c330-43c3-8bbb-255e8e77a469 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:35.654556 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a779bd-c330-43c3-8bbb-255e8e77a469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:35.672041 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-c1a779bd-c330-43c3-8bbb-255e8e77a469 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:35.673931 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c1a779bd-c330-43c3-8bbb-255e8e77a469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:35.705266 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1410/2798] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:02:35.717862 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd436802-5242-49fd-abff-0a81c54d0cb9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:35.719399 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dd436802-5242-49fd-abff-0a81c54d0cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:35.735988 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1389/2799] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:02:35.745723 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a779bd-c330-43c3-8bbb-255e8e77a469 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:35.747257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-c1a779bd-c330-43c3-8bbb-255e8e77a469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:35.778785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd436802-5242-49fd-abff-0a81c54d0cb9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:35.799706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:35.807355 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a779bd-c330-43c3-8bbb-255e8e77a469 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:35.811470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bfba0-c31f-400c-9038-3ef8de6a15da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:35.812121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bfba0-c31f-400c-9038-3ef8de6a15da service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:35.812414 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bfba0-c31f-400c-9038-3ef8de6a15da service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:35.812690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bfba0-c31f-400c-9038-3ef8de6a15da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:35.823459 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bfba0-c31f-400c-9038-3ef8de6a15da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:35.825393 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bfba0-c31f-400c-9038-3ef8de6a15da service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:35.830621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bcd5f-4fa2-4716-a71e-4b70c08d160c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:35.831289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bcd5f-4fa2-4716-a71e-4b70c08d160c service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:35.831584 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bcd5f-4fa2-4716-a71e-4b70c08d160c service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:35.831860 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bcd5f-4fa2-4716-a71e-4b70c08d160c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:35.835525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1390/2800] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:02:35.898945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bcd5f-4fa2-4716-a71e-4b70c08d160c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:35.900397 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bcd5f-4fa2-4716-a71e-4b70c08d160c service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:35.911277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1411/2801] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:02:36.207787 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bfba0-c31f-400c-9038-3ef8de6a15da service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:36.209265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-699bfba0-c31f-400c-9038-3ef8de6a15da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:36.224175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bfba0-c31f-400c-9038-3ef8de6a15da service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:36.228920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:36.229804 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:36.230084 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:36.230361 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] PATH_INFO: `/v3/projects/31bf6345dcd5437fabe949c7c9c24b0c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:36.231560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31bf6345dcd5437fabe949c7c9c24b0c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:36.233639 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:36.236840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:36.378158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31bf6345dcd5437fabe949c7c9c24b0c', 'request_id': 'req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:36.384364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31bf6345dcd5437fabe949c7c9c24b0c', 'request_id': 'req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:36.389993 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31bf6345dcd5437fabe949c7c9c24b0c', 'request_id': 'req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:36.480756 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] The token cache is being invalidate because project 31bf6345dcd5437fabe949c7c9c24b0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:36.481006 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:36.481627 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:36.483281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1391/2802] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:36 2022] DELETE /identity/v3/projects/31bf6345dcd5437fabe949c7c9c24b0c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:36.487402 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bcd5f-4fa2-4716-a71e-4b70c08d160c service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:36.497993 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bcd5f-4fa2-4716-a71e-4b70c08d160c service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:36.557048 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:36.557949 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:36.558226 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:36.558496 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] PATH_INFO: `/v3/users/283b66e1059e4c85a67d6c1a1e24f35c` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:36.559681 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=283b66e1059e4c85a67d6c1a1e24f35c)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:36.561643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:36.751629 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '283b66e1059e4c85a67d6c1a1e24f35c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:36.763977 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '283b66e1059e4c85a67d6c1a1e24f35c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:36.771744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '283b66e1059e4c85a67d6c1a1e24f35c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:36.774272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:36.840009 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '283b66e1059e4c85a67d6c1a1e24f35c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:36.840980 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '283b66e1059e4c85a67d6c1a1e24f35c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:36.847394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1412/2803] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:36 2022] DELETE /identity/v3/users/283b66e1059e4c85a67d6c1a1e24f35c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:36.951510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:36.953018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:37.019598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43861d-8b72-4d3e-9bc4-f336e5c32e28 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:37.086260 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f7ac7-a7ad-4371-b704-c0cc29fb5153 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b9a63f2a1244eb94518db1838bd212', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6846e3d3c19a4b469c4667802e951067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:37.086935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f7ac7-a7ad-4371-b704-c0cc29fb5153 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:37.087221 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f7ac7-a7ad-4371-b704-c0cc29fb5153 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:37.087492 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f7ac7-a7ad-4371-b704-c0cc29fb5153 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:37.088224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f7ac7-a7ad-4371-b704-c0cc29fb5153 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:02:37.088532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f7ac7-a7ad-4371-b704-c0cc29fb5153 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:37.090377 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f7ac7-a7ad-4371-b704-c0cc29fb5153 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:37.103277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1392/2804] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:02:37.108240 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:37.109854 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:37.124562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118572fc-d18a-4b5e-ac32-1fa3e3e26fe3 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:37.130063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85552c0-d709-41da-b3b9-cbaafde3739b tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b9a63f2a1244eb94518db1838bd212', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6846e3d3c19a4b469c4667802e951067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:37.130943 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85552c0-d709-41da-b3b9-cbaafde3739b tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:37.131094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85552c0-d709-41da-b3b9-cbaafde3739b tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:37.131477 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85552c0-d709-41da-b3b9-cbaafde3739b tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:37.132008 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85552c0-d709-41da-b3b9-cbaafde3739b tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:37.133978 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85552c0-d709-41da-b3b9-cbaafde3739b tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:37.208304 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1413/2805] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:37 2022] POST /identity/v3/projects => generated 354 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:37.212403 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f7ac7-a7ad-4371-b704-c0cc29fb5153 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:37.223468 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f7ac7-a7ad-4371-b704-c0cc29fb5153 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:37.253878 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85552c0-d709-41da-b3b9-cbaafde3739b tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:37.256073 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:37.258388 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-d85552c0-d709-41da-b3b9-cbaafde3739b tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:37.291164 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fc6fd-1e20-4b0f-aff8-bac8e29cce49 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b9a63f2a1244eb94518db1838bd212', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6846e3d3c19a4b469c4667802e951067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:37.291753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc6fd-1e20-4b0f-aff8-bac8e29cce49 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:37.291753 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc6fd-1e20-4b0f-aff8-bac8e29cce49 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:37.292128 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fc6fd-1e20-4b0f-aff8-bac8e29cce49 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:37.292586 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fc6fd-1e20-4b0f-aff8-bac8e29cce49 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:02:37.292918 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fc6fd-1e20-4b0f-aff8-bac8e29cce49 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:37.294621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fc6fd-1e20-4b0f-aff8-bac8e29cce49 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:37.308230 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1393/2806] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:02:37.347090 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fc6fd-1e20-4b0f-aff8-bac8e29cce49 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:37.347644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85552c0-d709-41da-b3b9-cbaafde3739b tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:37.352615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:37.353511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:37.353798 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:37.354104 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] PATH_INFO: `/v3/projects/4b69745eb9164f8da36d9489162207d7` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:37.355306 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b69745eb9164f8da36d9489162207d7)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:37.357335 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fc6fd-1e20-4b0f-aff8-bac8e29cce49 tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:37.357481 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:37.363520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15b9a63f2a1244eb94518db1838bd212', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6846e3d3c19a4b469c4667802e951067', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:37.367776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:37.367776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:37.367776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] PATH_INFO: `/v3/projects/4742a7df1e944aa68450a1ca54b7cd69` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:37.367776 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4742a7df1e944aa68450a1ca54b7cd69)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:37.368178 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:37.486312 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b69745eb9164f8da36d9489162207d7', 'request_id': 'req-8ed96e6f-77c1-4670-ba1e-9f862cb11738'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:37.508470 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4742a7df1e944aa68450a1ca54b7cd69', 'request_id': 'req-176053bf-d160-4bee-94b7-f55aa36b7d8c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:37.514759 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4742a7df1e944aa68450a1ca54b7cd69', 'request_id': 'req-176053bf-d160-4bee-94b7-f55aa36b7d8c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:37.520868 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4742a7df1e944aa68450a1ca54b7cd69', 'request_id': 'req-176053bf-d160-4bee-94b7-f55aa36b7d8c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:37.554427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b69745eb9164f8da36d9489162207d7', 'request_id': 'req-8ed96e6f-77c1-4670-ba1e-9f862cb11738'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:37.560963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b69745eb9164f8da36d9489162207d7', 'request_id': 'req-8ed96e6f-77c1-4670-ba1e-9f862cb11738'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:37.573526 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] The token cache is being invalidate because project 4b69745eb9164f8da36d9489162207d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:37.573820 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:37.574601 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-8ed96e6f-77c1-4670-ba1e-9f862cb11738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:37.576170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1414/2807] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:37 2022] DELETE /identity/v3/projects/4b69745eb9164f8da36d9489162207d7 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:02:37.581833 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03524f39-a558-4bdc-81c9-d9803855d676 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:37.582120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03524f39-a558-4bdc-81c9-d9803855d676 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:37.582398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03524f39-a558-4bdc-81c9-d9803855d676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:37.600392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-03524f39-a558-4bdc-81c9-d9803855d676 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:37.601627 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-03524f39-a558-4bdc-81c9-d9803855d676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:37.628520 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] The token cache is being invalidate because project 4742a7df1e944aa68450a1ca54b7cd69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:37.628806 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:37.629493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-176053bf-d160-4bee-94b7-f55aa36b7d8c tempest-VolumeQuotaClassesTest-1695102113 tempest-VolumeQuotaClassesTest-1695102113-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:37.631158 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1394/2808] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:37 2022] DELETE /identity/v3/projects/4742a7df1e944aa68450a1ca54b7cd69 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:37.662605 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1415/2809] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:37.670311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9727-0a0e-4483-9bce-6cf5bcb21523 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:37.673107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9727-0a0e-4483-9bce-6cf5bcb21523 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:37.673107 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9727-0a0e-4483-9bce-6cf5bcb21523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:37.690883 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-518b9727-0a0e-4483-9bce-6cf5bcb21523 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:37.692106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-518b9727-0a0e-4483-9bce-6cf5bcb21523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:37.700094 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1395/2810] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:37.705282 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03524f39-a558-4bdc-81c9-d9803855d676 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:37.710228 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:37.715719 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03524f39-a558-4bdc-81c9-d9803855d676 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:37.720550 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969ffed-df05-454b-9ead-73ec3a4b16eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:37.721247 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969ffed-df05-454b-9ead-73ec3a4b16eb admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:37.721530 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969ffed-df05-454b-9ead-73ec3a4b16eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:37.721812 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969ffed-df05-454b-9ead-73ec3a4b16eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:37.722542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969ffed-df05-454b-9ead-73ec3a4b16eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:02:37.722847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969ffed-df05-454b-9ead-73ec3a4b16eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:37.724453 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969ffed-df05-454b-9ead-73ec3a4b16eb admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:37.738581 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1416/2811] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:02:37.743133 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b9727-0a0e-4483-9bce-6cf5bcb21523 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:37.755021 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b9727-0a0e-4483-9bce-6cf5bcb21523 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:37.759744 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826603e-e7d9-4174-86a1-2d77423bacce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:37.760395 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826603e-e7d9-4174-86a1-2d77423bacce admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:37.760684 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826603e-e7d9-4174-86a1-2d77423bacce admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:37.760990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826603e-e7d9-4174-86a1-2d77423bacce admin admin] PATH_INFO: `/v3/projects` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:37.761792 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826603e-e7d9-4174-86a1-2d77423bacce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:37.764020 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826603e-e7d9-4174-86a1-2d77423bacce admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:37.865201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1396/2812] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:37 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:37.869371 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969ffed-df05-454b-9ead-73ec3a4b16eb admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:37.880266 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969ffed-df05-454b-9ead-73ec3a4b16eb admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:37.936133 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa91080-f4be-4691-a8fe-e2af0962ed9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:37.937239 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa91080-f4be-4691-a8fe-e2af0962ed9d admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:37.937600 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa91080-f4be-4691-a8fe-e2af0962ed9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:37.938088 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa91080-f4be-4691-a8fe-e2af0962ed9d admin admin] PATH_INFO: `/v3/users` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:37.940230 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa91080-f4be-4691-a8fe-e2af0962ed9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:37.942219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa91080-f4be-4691-a8fe-e2af0962ed9d admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:38.023344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1417/2813] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:37 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:38.028935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826603e-e7d9-4174-86a1-2d77423bacce admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:38.039648 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826603e-e7d9-4174-86a1-2d77423bacce admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:38.044421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecfb7d-7811-4d22-8c27-66aed850bd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:38.045236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecfb7d-7811-4d22-8c27-66aed850bd28 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:38.045236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecfb7d-7811-4d22-8c27-66aed850bd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:38.050217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecfb7d-7811-4d22-8c27-66aed850bd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:38.050217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecfb7d-7811-4d22-8c27-66aed850bd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:38.050217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecfb7d-7811-4d22-8c27-66aed850bd28 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:38.059453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1397/2814] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:38.064531 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa91080-f4be-4691-a8fe-e2af0962ed9d admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:38.077002 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa91080-f4be-4691-a8fe-e2af0962ed9d admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:38.082079 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c21067-3a4f-462b-b57f-b578ae58ab17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:38.082748 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c21067-3a4f-462b-b57f-b578ae58ab17 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:38.083035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c21067-3a4f-462b-b57f-b578ae58ab17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:38.083313 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c21067-3a4f-462b-b57f-b578ae58ab17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:38.084082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c21067-3a4f-462b-b57f-b578ae58ab17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:38.085731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c21067-3a4f-462b-b57f-b578ae58ab17 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:38.096469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1418/2815] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:38.102046 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecfb7d-7811-4d22-8c27-66aed850bd28 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:38.114933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecfb7d-7811-4d22-8c27-66aed850bd28 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:38.120100 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384e432-218f-4b4c-a0d4-6734c867306c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:38.120935 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384e432-218f-4b4c-a0d4-6734c867306c admin admin] REQUEST_METHOD: `PUT` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:38.121226 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384e432-218f-4b4c-a0d4-6734c867306c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:38.121504 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384e432-218f-4b4c-a0d4-6734c867306c admin admin] PATH_INFO: `/v3/projects/73a4c19197794a55875f263b35f4dd91/users/5b81ac157ef94f598c0ea7d19ac65a45/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:38.138147 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384e432-218f-4b4c-a0d4-6734c867306c admin admin] RBAC: Authorizing `identity:create_grant(project_id=73a4c19197794a55875f263b35f4dd91, user_id=5b81ac157ef94f598c0ea7d19ac65a45, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:38.140795 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384e432-218f-4b4c-a0d4-6734c867306c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:38.234693 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1398/2816] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:38 2022] PUT /identity/v3/projects/73a4c19197794a55875f263b35f4dd91/users/5b81ac157ef94f598c0ea7d19ac65a45/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:38.245699 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a944b-6f7b-4da9-948f-f2c8d67d9b01 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:38.245998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a944b-6f7b-4da9-948f-f2c8d67d9b01 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:38.246289 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a944b-6f7b-4da9-948f-f2c8d67d9b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:38.263426 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384e432-218f-4b4c-a0d4-6734c867306c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:38.265194 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a384e432-218f-4b4c-a0d4-6734c867306c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:38.276937 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-b14a944b-6f7b-4da9-948f-f2c8d67d9b01 None None] MFA Rules not processed for user `5b81ac157ef94f598c0ea7d19ac65a45`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:38.278154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b14a944b-6f7b-4da9-948f-f2c8d67d9b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:38.317685 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:38.335201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384e432-218f-4b4c-a0d4-6734c867306c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:38.340169 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:38.341075 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:38.341362 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:38.341643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] PATH_INFO: `/v3/users/cb331830c4f847c4ade846d0e2698e1c` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:38.343048 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb331830c4f847c4ade846d0e2698e1c)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:38.345021 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:38.345754 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1419/2817] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:38.356891 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a944b-6f7b-4da9-948f-f2c8d67d9b01 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:38.359416 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-b14a944b-6f7b-4da9-948f-f2c8d67d9b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:38.417112 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a944b-6f7b-4da9-948f-f2c8d67d9b01 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:38.420716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db07bd-af65-4749-ae73-83d8ebf97873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:38.421437 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db07bd-af65-4749-ae73-83d8ebf97873 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:38.421718 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db07bd-af65-4749-ae73-83d8ebf97873 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:38.421996 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db07bd-af65-4749-ae73-83d8ebf97873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:38.431111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db07bd-af65-4749-ae73-83d8ebf97873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:38.432562 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db07bd-af65-4749-ae73-83d8ebf97873 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:38.437413 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:38.442895 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1420/2818] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:38.528859 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb331830c4f847c4ade846d0e2698e1c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:38.535933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb331830c4f847c4ade846d0e2698e1c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:38.549115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb331830c4f847c4ade846d0e2698e1c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:38.634319 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb331830c4f847c4ade846d0e2698e1c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:38.634978 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb331830c4f847c4ade846d0e2698e1c'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:38.641557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1399/2819] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:38 2022] DELETE /identity/v3/users/cb331830c4f847c4ade846d0e2698e1c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 17:02:38.646042 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a5df8-ba28-4c78-9d62-be884b89247c None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:38.646498 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a5df8-ba28-4c78-9d62-be884b89247c None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:38.646621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a5df8-ba28-4c78-9d62-be884b89247c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:38.667332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-895a5df8-ba28-4c78-9d62-be884b89247c None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:38.668170 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-895a5df8-ba28-4c78-9d62-be884b89247c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:38.731764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1421/2820] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:38.742357 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:38.743775 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:38.802501 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ae710-aa1a-466f-9744-0781240e73bf admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:38.806265 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae76dd-29cd-4fa4-af22-2eb61fe2dd0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:38.806914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae76dd-29cd-4fa4-af22-2eb61fe2dd0a service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:38.807199 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae76dd-29cd-4fa4-af22-2eb61fe2dd0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:38.807478 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae76dd-29cd-4fa4-af22-2eb61fe2dd0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:38.817700 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae76dd-29cd-4fa4-af22-2eb61fe2dd0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:38.819237 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae76dd-29cd-4fa4-af22-2eb61fe2dd0a service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:38.829079 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1400/2821] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:39.619269 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a5df8-ba28-4c78-9d62-be884b89247c None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:39.620859 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-895a5df8-ba28-4c78-9d62-be884b89247c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:39.634769 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a5df8-ba28-4c78-9d62-be884b89247c None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:39.641138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:39.642091 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:39.642302 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:39.642610 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] PATH_INFO: `/v3/projects/e0316cef126347c1bc5b46bba46ddb06` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:39.643790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0316cef126347c1bc5b46bba46ddb06)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:39.645883 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:39.767618 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0316cef126347c1bc5b46bba46ddb06', 'request_id': 'req-75ab830b-ebd9-4925-b261-4e907ca6e82c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:39.822662 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0316cef126347c1bc5b46bba46ddb06', 'request_id': 'req-75ab830b-ebd9-4925-b261-4e907ca6e82c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:39.828997 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0316cef126347c1bc5b46bba46ddb06', 'request_id': 'req-75ab830b-ebd9-4925-b261-4e907ca6e82c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:39.841287 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] The token cache is being invalidate because project e0316cef126347c1bc5b46bba46ddb06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:39.841607 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:39.842278 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:39.843838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1422/2822] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:39 2022] DELETE /identity/v3/projects/e0316cef126347c1bc5b46bba46ddb06 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:39.847933 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae76dd-29cd-4fa4-af22-2eb61fe2dd0a service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:39.857676 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae76dd-29cd-4fa4-af22-2eb61fe2dd0a service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:39.919394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259d644-68a8-4ecf-8f70-221caa7451c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:39.920282 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259d644-68a8-4ecf-8f70-221caa7451c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:39.920562 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259d644-68a8-4ecf-8f70-221caa7451c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:39.920835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259d644-68a8-4ecf-8f70-221caa7451c7 admin admin] PATH_INFO: `/v3/users/15b9a63f2a1244eb94518db1838bd212` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:39.922126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259d644-68a8-4ecf-8f70-221caa7451c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15b9a63f2a1244eb94518db1838bd212)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:39.924106 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259d644-68a8-4ecf-8f70-221caa7451c7 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:39.935103 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:40.109688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0259d644-68a8-4ecf-8f70-221caa7451c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15b9a63f2a1244eb94518db1838bd212'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:40.117003 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0259d644-68a8-4ecf-8f70-221caa7451c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15b9a63f2a1244eb94518db1838bd212'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:40.130053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0259d644-68a8-4ecf-8f70-221caa7451c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15b9a63f2a1244eb94518db1838bd212'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:40.222275 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0259d644-68a8-4ecf-8f70-221caa7451c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15b9a63f2a1244eb94518db1838bd212'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:40.222921 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-0259d644-68a8-4ecf-8f70-221caa7451c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15b9a63f2a1244eb94518db1838bd212'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:40.229902 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1401/2823] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:39 2022] DELETE /identity/v3/users/15b9a63f2a1244eb94518db1838bd212 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:40.705876 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:40.708660 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:40.724467 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab830b-ebd9-4925-b261-4e907ca6e82c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:40.789105 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:40.789968 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:40.790246 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:40.790519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] PATH_INFO: `/v3/projects/6846e3d3c19a4b469c4667802e951067` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:40.791695 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6846e3d3c19a4b469c4667802e951067)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:40.793641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:40.881400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:40.899154 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6846e3d3c19a4b469c4667802e951067', 'request_id': 'req-881fe339-806a-4b7f-8daa-8fd08572d11c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:40.954377 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6846e3d3c19a4b469c4667802e951067', 'request_id': 'req-881fe339-806a-4b7f-8daa-8fd08572d11c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:40.961004 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6846e3d3c19a4b469c4667802e951067', 'request_id': 'req-881fe339-806a-4b7f-8daa-8fd08572d11c'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:40.974111 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] The token cache is being invalidate because project 6846e3d3c19a4b469c4667802e951067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:40.974401 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:40.975065 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-881fe339-806a-4b7f-8daa-8fd08572d11c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:40.976775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1423/2824] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:40 2022] DELETE /identity/v3/projects/6846e3d3c19a4b469c4667802e951067 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:40.981680 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a34c55-d375-458a-805a-ed30d470c1b9 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:40.982168 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a34c55-d375-458a-805a-ed30d470c1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:40.985004 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a34c55-d375-458a-805a-ed30d470c1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:41.003005 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-e6a34c55-d375-458a-805a-ed30d470c1b9 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:41.004284 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-e6a34c55-d375-458a-805a-ed30d470c1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:41.062201 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1402/2825] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:41.070828 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c841150-d86d-4562-a7f9-d800f6dcc958 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:41.071125 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c841150-d86d-4562-a7f9-d800f6dcc958 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:41.071405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c841150-d86d-4562-a7f9-d800f6dcc958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:41.089715 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-8c841150-d86d-4562-a7f9-d800f6dcc958 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:41.090940 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-8c841150-d86d-4562-a7f9-d800f6dcc958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:41.098910 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1424/2826] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:41.103086 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a34c55-d375-458a-805a-ed30d470c1b9 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:41.113779 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a34c55-d375-458a-805a-ed30d470c1b9 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:41.118399 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74b6d4-a194-40cc-a57b-6ecc519c43d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:41.119057 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74b6d4-a194-40cc-a57b-6ecc519c43d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:41.119343 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74b6d4-a194-40cc-a57b-6ecc519c43d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:41.119621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74b6d4-a194-40cc-a57b-6ecc519c43d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:41.120346 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74b6d4-a194-40cc-a57b-6ecc519c43d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:02:41.120653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74b6d4-a194-40cc-a57b-6ecc519c43d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:41.122286 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74b6d4-a194-40cc-a57b-6ecc519c43d5 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:41.134725 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1403/2827] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:02:41.138902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c841150-d86d-4562-a7f9-d800f6dcc958 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:41.148171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c841150-d86d-4562-a7f9-d800f6dcc958 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:41.153032 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31621c30-a235-4401-a3c8-b5d35f156cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:41.153684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31621c30-a235-4401-a3c8-b5d35f156cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:41.153969 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31621c30-a235-4401-a3c8-b5d35f156cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:41.154249 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31621c30-a235-4401-a3c8-b5d35f156cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:41.154946 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31621c30-a235-4401-a3c8-b5d35f156cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:41.156851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31621c30-a235-4401-a3c8-b5d35f156cb5 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:41.272055 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1425/2828] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:41 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:41.276360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74b6d4-a194-40cc-a57b-6ecc519c43d5 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:41.287674 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74b6d4-a194-40cc-a57b-6ecc519c43d5 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:41.355810 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019c780-30fb-4326-9e89-24ece355831b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:41.356491 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019c780-30fb-4326-9e89-24ece355831b admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:41.356771 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019c780-30fb-4326-9e89-24ece355831b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:41.357091 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019c780-30fb-4326-9e89-24ece355831b admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:41.357767 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019c780-30fb-4326-9e89-24ece355831b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:41.359688 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019c780-30fb-4326-9e89-24ece355831b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:41.446601 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1404/2829] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:41 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:41.450611 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31621c30-a235-4401-a3c8-b5d35f156cb5 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:41.464676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31621c30-a235-4401-a3c8-b5d35f156cb5 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:41.469495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77726fa9-7c10-4da6-96d0-fd65ed3d40d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:41.470647 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77726fa9-7c10-4da6-96d0-fd65ed3d40d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:41.470933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77726fa9-7c10-4da6-96d0-fd65ed3d40d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:41.471214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77726fa9-7c10-4da6-96d0-fd65ed3d40d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:41.471951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77726fa9-7c10-4da6-96d0-fd65ed3d40d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:41.473632 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77726fa9-7c10-4da6-96d0-fd65ed3d40d9 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:41.484945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1426/2830] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:41.489673 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019c780-30fb-4326-9e89-24ece355831b admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:41.496378 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:41.500255 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019c780-30fb-4326-9e89-24ece355831b admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:41.504968 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983ebc7-d3c8-4ed1-b899-52273ec9d7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:41.505629 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983ebc7-d3c8-4ed1-b899-52273ec9d7ee admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:41.505914 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983ebc7-d3c8-4ed1-b899-52273ec9d7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:41.506193 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983ebc7-d3c8-4ed1-b899-52273ec9d7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:41.506922 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983ebc7-d3c8-4ed1-b899-52273ec9d7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:41.508525 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0983ebc7-d3c8-4ed1-b899-52273ec9d7ee admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:41.519620 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1405/2831] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:41.524131 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77726fa9-7c10-4da6-96d0-fd65ed3d40d9 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:41.534734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77726fa9-7c10-4da6-96d0-fd65ed3d40d9 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:41.539641 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672f3e4-d5a3-4be7-8421-05037d214d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:41.540519 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672f3e4-d5a3-4be7-8421-05037d214d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:41.540803 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672f3e4-d5a3-4be7-8421-05037d214d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:41.541113 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672f3e4-d5a3-4be7-8421-05037d214d63 admin admin] PATH_INFO: `/v3/projects/309e6eabc77347cbbfd9c400084617d5/users/d9627a1fc527463ba2a69cfd1bd55634/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:41.556536 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672f3e4-d5a3-4be7-8421-05037d214d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=309e6eabc77347cbbfd9c400084617d5, user_id=d9627a1fc527463ba2a69cfd1bd55634, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:41.559349 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672f3e4-d5a3-4be7-8421-05037d214d63 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:41.640171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1427/2832] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:41 2022] PUT /identity/v3/projects/309e6eabc77347cbbfd9c400084617d5/users/d9627a1fc527463ba2a69cfd1bd55634/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:41.652222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa98574-f973-408d-a42d-8610cbcc79ad None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:41.652994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa98574-f973-408d-a42d-8610cbcc79ad None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:41.652994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa98574-f973-408d-a42d-8610cbcc79ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:41.679975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-aaa98574-f973-408d-a42d-8610cbcc79ad None None] MFA Rules not processed for user `d9627a1fc527463ba2a69cfd1bd55634`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:41.681225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-aaa98574-f973-408d-a42d-8610cbcc79ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:41.745616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1406/2833] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:02:41.756928 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672f3e4-d5a3-4be7-8421-05037d214d63 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:41.758742 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4672f3e4-d5a3-4be7-8421-05037d214d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:41.815469 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672f3e4-d5a3-4be7-8421-05037d214d63 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:41.819094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480b4b8-53ec-4001-a872-e12197e7cea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:41.819731 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480b4b8-53ec-4001-a872-e12197e7cea9 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:41.820016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480b4b8-53ec-4001-a872-e12197e7cea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:41.820288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480b4b8-53ec-4001-a872-e12197e7cea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:41.828527 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480b4b8-53ec-4001-a872-e12197e7cea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:41.830495 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480b4b8-53ec-4001-a872-e12197e7cea9 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:41.839504 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1428/2834] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:41.859012 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa98574-f973-408d-a42d-8610cbcc79ad None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:41.922568 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa98574-f973-408d-a42d-8610cbcc79ad None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:41.927627 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:41.928570 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:41.928909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:41.929191 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] PATH_INFO: `/v3/users/d9627a1fc527463ba2a69cfd1bd55634` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:41.930427 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9627a1fc527463ba2a69cfd1bd55634)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:41.932366 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:42.128017 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9627a1fc527463ba2a69cfd1bd55634'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:42.135296 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9627a1fc527463ba2a69cfd1bd55634'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:42.148000 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9627a1fc527463ba2a69cfd1bd55634'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:42.211537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9627a1fc527463ba2a69cfd1bd55634'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:42.211537 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9627a1fc527463ba2a69cfd1bd55634'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:42.218045 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1407/2835] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:02:41 2022] DELETE /identity/v3/users/d9627a1fc527463ba2a69cfd1bd55634 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:42.223171 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cf305-c453-4216-bf2e-7b0c2693a959 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:42.225394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cf305-c453-4216-bf2e-7b0c2693a959 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:42.225394 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cf305-c453-4216-bf2e-7b0c2693a959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:42.242338 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-3f5cf305-c453-4216-bf2e-7b0c2693a959 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:42.243523 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3f5cf305-c453-4216-bf2e-7b0c2693a959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:42.292579 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:42.306020 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1429/2836] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:42.316515 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:42.318149 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:42.323826 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:42.398440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1c5f6-f8ae-4cbe-956b-a8c7dcb3a381 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:42.403600 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765e8d-7643-4a20-8f41-b11febf76298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:42.404421 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765e8d-7643-4a20-8f41-b11febf76298 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:42.404852 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765e8d-7643-4a20-8f41-b11febf76298 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:42.405321 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765e8d-7643-4a20-8f41-b11febf76298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:42.419532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b765e8d-7643-4a20-8f41-b11febf76298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:42.421058 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b765e8d-7643-4a20-8f41-b11febf76298 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:42.431279 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1408/2837] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:42.813903 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cf305-c453-4216-bf2e-7b0c2693a959 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:42.815691 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-3f5cf305-c453-4216-bf2e-7b0c2693a959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:42.828359 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cf305-c453-4216-bf2e-7b0c2693a959 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:42.833713 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:42.834587 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:42.834917 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:42.835191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] PATH_INFO: `/v3/projects/309e6eabc77347cbbfd9c400084617d5` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:42.836405 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=309e6eabc77347cbbfd9c400084617d5)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:42.838446 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:42.943345 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '309e6eabc77347cbbfd9c400084617d5', 'request_id': 'req-0d75b950-b6ff-4c89-b290-2d3aba5bad22'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:43.016160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '309e6eabc77347cbbfd9c400084617d5', 'request_id': 'req-0d75b950-b6ff-4c89-b290-2d3aba5bad22'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:43.022941 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '309e6eabc77347cbbfd9c400084617d5', 'request_id': 'req-0d75b950-b6ff-4c89-b290-2d3aba5bad22'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:43.034163 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] The token cache is being invalidate because project 309e6eabc77347cbbfd9c400084617d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:02:43.034452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:43.035085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-0d75b950-b6ff-4c89-b290-2d3aba5bad22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:02:43.036633 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1430/2838] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:02:42 2022] DELETE /identity/v3/projects/309e6eabc77347cbbfd9c400084617d5 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:43.041690 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3fff6-fa25-41ea-aa77-d7d83f19c8f5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:43.041982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3fff6-fa25-41ea-aa77-d7d83f19c8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:43.042259 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3fff6-fa25-41ea-aa77-d7d83f19c8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:43.059991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a4e3fff6-fa25-41ea-aa77-d7d83f19c8f5 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:43.069387 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a4e3fff6-fa25-41ea-aa77-d7d83f19c8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:43.121803 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1409/2839] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:43.180370 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d59dc-f40d-41ec-9155-f456c1666e22 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:43.180591 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d59dc-f40d-41ec-9155-f456c1666e22 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:43.180844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d59dc-f40d-41ec-9155-f456c1666e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:43.200083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-989d59dc-f40d-41ec-9155-f456c1666e22 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:43.201354 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-989d59dc-f40d-41ec-9155-f456c1666e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:43.210192 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1431/2840] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:43.214408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3fff6-fa25-41ea-aa77-d7d83f19c8f5 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:43.224752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3fff6-fa25-41ea-aa77-d7d83f19c8f5 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:43.229524 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa0a6c-902e-4d8e-85c8-43bb8d2aad0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:43.230171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa0a6c-902e-4d8e-85c8-43bb8d2aad0c admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:43.230451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa0a6c-902e-4d8e-85c8-43bb8d2aad0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:43.230742 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa0a6c-902e-4d8e-85c8-43bb8d2aad0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:43.231447 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa0a6c-902e-4d8e-85c8-43bb8d2aad0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:02:43.231748 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa0a6c-902e-4d8e-85c8-43bb8d2aad0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:43.233398 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa0a6c-902e-4d8e-85c8-43bb8d2aad0c admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:43.245171 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1410/2841] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:02:43.250052 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d59dc-f40d-41ec-9155-f456c1666e22 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:43.259366 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d59dc-f40d-41ec-9155-f456c1666e22 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:43.264144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a3052-ad20-4ea1-9c00-15cbb2ff7bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:43.264797 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a3052-ad20-4ea1-9c00-15cbb2ff7bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:43.265120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a3052-ad20-4ea1-9c00-15cbb2ff7bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:43.265400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a3052-ad20-4ea1-9c00-15cbb2ff7bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:43.266087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a3052-ad20-4ea1-9c00-15cbb2ff7bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:43.272059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a3052-ad20-4ea1-9c00-15cbb2ff7bc0 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:43.372934 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1432/2842] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:02:43 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:43.377352 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa0a6c-902e-4d8e-85c8-43bb8d2aad0c admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:43.387560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa0a6c-902e-4d8e-85c8-43bb8d2aad0c admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:43.446246 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb12bc-6e2d-471c-a284-cf03419ae4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:43.446858 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb12bc-6e2d-471c-a284-cf03419ae4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:43.447144 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb12bc-6e2d-471c-a284-cf03419ae4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:43.447423 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb12bc-6e2d-471c-a284-cf03419ae4b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:43.448129 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb12bc-6e2d-471c-a284-cf03419ae4b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:43.450098 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb12bc-6e2d-471c-a284-cf03419ae4b2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:43.526056 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1411/2843] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:02:43 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:02:43.530120 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a3052-ad20-4ea1-9c00-15cbb2ff7bc0 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:43.540634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a3052-ad20-4ea1-9c00-15cbb2ff7bc0 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:43.545367 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91d864-c038-48c1-88cf-64f45eb18eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:43.546035 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91d864-c038-48c1-88cf-64f45eb18eba admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:43.546320 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91d864-c038-48c1-88cf-64f45eb18eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:43.546597 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91d864-c038-48c1-88cf-64f45eb18eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:43.547326 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91d864-c038-48c1-88cf-64f45eb18eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:43.548951 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e91d864-c038-48c1-88cf-64f45eb18eba admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:43.560551 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1433/2844] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:43.567587 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb12bc-6e2d-471c-a284-cf03419ae4b2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:43.577895 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb12bc-6e2d-471c-a284-cf03419ae4b2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:43.583375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6eeb17-4ad5-47f9-a24a-51475ca09fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:43.584030 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eeb17-4ad5-47f9-a24a-51475ca09fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:43.584306 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eeb17-4ad5-47f9-a24a-51475ca09fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:43.584581 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eeb17-4ad5-47f9-a24a-51475ca09fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:43.585337 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6eeb17-4ad5-47f9-a24a-51475ca09fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:43.586977 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6eeb17-4ad5-47f9-a24a-51475ca09fa6 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:43.596866 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1412/2845] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:02:43.600954 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91d864-c038-48c1-88cf-64f45eb18eba admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:43.611684 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91d864-c038-48c1-88cf-64f45eb18eba admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:43.616685 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec8e7a-c873-4bf4-b010-58e03211245c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:43.617592 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec8e7a-c873-4bf4-b010-58e03211245c admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:43.617872 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec8e7a-c873-4bf4-b010-58e03211245c admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:43.618144 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec8e7a-c873-4bf4-b010-58e03211245c admin admin] PATH_INFO: `/v3/projects/18f27a3568f447279e227939696730bf/users/6daaa1076f054dd39fdf03017adb0d2f/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:43.634568 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec8e7a-c873-4bf4-b010-58e03211245c admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f27a3568f447279e227939696730bf, user_id=6daaa1076f054dd39fdf03017adb0d2f, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:43.637298 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec8e7a-c873-4bf4-b010-58e03211245c admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:43.732081 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1434/2846] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:02:43 2022] PUT /identity/v3/projects/18f27a3568f447279e227939696730bf/users/6daaa1076f054dd39fdf03017adb0d2f/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:02:43.740809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67d493-b22f-495d-9705-cbda603912c2 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:43.740809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67d493-b22f-495d-9705-cbda603912c2 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:43.740809 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67d493-b22f-495d-9705-cbda603912c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:43.757050 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-8a67d493-b22f-495d-9705-cbda603912c2 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:43.758312 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-8a67d493-b22f-495d-9705-cbda603912c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:43.763486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:43.822236 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1413/2847] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:43.832343 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec8e7a-c873-4bf4-b010-58e03211245c admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:43.834266 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-27ec8e7a-c873-4bf4-b010-58e03211245c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:43.892532 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec8e7a-c873-4bf4-b010-58e03211245c admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:43.896934 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8227a-2d82-4e38-bb8b-8d79b248d5a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:43.897583 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8227a-2d82-4e38-bb8b-8d79b248d5a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:43.897865 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8227a-2d82-4e38-bb8b-8d79b248d5a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:43.898139 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8227a-2d82-4e38-bb8b-8d79b248d5a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:43.907775 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8227a-2d82-4e38-bb8b-8d79b248d5a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:43.909811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8227a-2d82-4e38-bb8b-8d79b248d5a9 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:43.919288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1435/2848] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:45.120686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496e1bd-bfdb-4c4e-9a97-bc899b832396 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:45.121292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496e1bd-bfdb-4c4e-9a97-bc899b832396 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:45.121292 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496e1bd-bfdb-4c4e-9a97-bc899b832396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:45.271957 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-0496e1bd-bfdb-4c4e-9a97-bc899b832396 None None] MFA Rules not processed for user `6daaa1076f054dd39fdf03017adb0d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:45.275954 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-0496e1bd-bfdb-4c4e-9a97-bc899b832396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:45.345996 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1414/2849] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:45.357673 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8227a-2d82-4e38-bb8b-8d79b248d5a9 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:45.359217 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-50a8227a-2d82-4e38-bb8b-8d79b248d5a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:45.378603 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:45.418331 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8227a-2d82-4e38-bb8b-8d79b248d5a9 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:45.421955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e2523-160a-45d0-990f-845ead9e7395 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:45.424634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e2523-160a-45d0-990f-845ead9e7395 service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:45.424634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e2523-160a-45d0-990f-845ead9e7395 service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:45.424634 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e2523-160a-45d0-990f-845ead9e7395 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:45.431931 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e2523-160a-45d0-990f-845ead9e7395 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:45.433432 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e2523-160a-45d0-990f-845ead9e7395 service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:45.443628 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1436/2850] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:48.160760 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955adaa-bd72-4d33-99f1-415851dd221e None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:48.161344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955adaa-bd72-4d33-99f1-415851dd221e None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:48.161344 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955adaa-bd72-4d33-99f1-415851dd221e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:48.179247 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-d955adaa-bd72-4d33-99f1-415851dd221e None None] MFA Rules not processed for user `6daaa1076f054dd39fdf03017adb0d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:02:48.180428 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d955adaa-bd72-4d33-99f1-415851dd221e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:48.187283 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1415/2851] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:02:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:02:48.424553 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e2523-160a-45d0-990f-845ead9e7395 service cinder] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:48.436283 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e2523-160a-45d0-990f-845ead9e7395 service cinder] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:48.440427 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e082a-55e0-4bdc-9df1-a01a30e6a4e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:48.441301 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e082a-55e0-4bdc-9df1-a01a30e6a4e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:48.441588 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e082a-55e0-4bdc-9df1-a01a30e6a4e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:48.441874 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e082a-55e0-4bdc-9df1-a01a30e6a4e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:48.443700 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bb4e082a-55e0-4bdc-9df1-a01a30e6a4e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:48.453814 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e082a-55e0-4bdc-9df1-a01a30e6a4e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:48.455257 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e082a-55e0-4bdc-9df1-a01a30e6a4e1 service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:48.464245 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1437/2852] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:48.841791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955adaa-bd72-4d33-99f1-415851dd221e None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:02:48.841791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-d955adaa-bd72-4d33-99f1-415851dd221e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:48.937590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955adaa-bd72-4d33-99f1-415851dd221e None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:02:48.943850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e16d8-eb0a-461b-af3f-4c52e3f071af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:02:48.944518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e16d8-eb0a-461b-af3f-4c52e3f071af service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:02:48.945042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e16d8-eb0a-461b-af3f-4c52e3f071af service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:02:48.946152 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e16d8-eb0a-461b-af3f-4c52e3f071af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:02:48.948108 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-519e16d8-eb0a-461b-af3f-4c52e3f071af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:02:48.962145 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e16d8-eb0a-461b-af3f-4c52e3f071af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:02:48.963765 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e16d8-eb0a-461b-af3f-4c52e3f071af service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:02:48.974907 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1416/2853] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:02:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:02:58.464784 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=25 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:02:58.969955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:01.721216 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e082a-55e0-4bdc-9df1-a01a30e6a4e1 service neutron] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:01.722643 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-bb4e082a-55e0-4bdc-9df1-a01a30e6a4e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:01.736580 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e082a-55e0-4bdc-9df1-a01a30e6a4e1 service neutron] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:01.741678 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122fcff-33a1-4e66-b390-674ae920eb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:01.742542 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122fcff-33a1-4e66-b390-674ae920eb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:01.742827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122fcff-33a1-4e66-b390-674ae920eb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:01.743107 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122fcff-33a1-4e66-b390-674ae920eb7b admin admin] PATH_INFO: `/v3/users/5b81ac157ef94f598c0ea7d19ac65a45` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:01.744315 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122fcff-33a1-4e66-b390-674ae920eb7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b81ac157ef94f598c0ea7d19ac65a45)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:01.746292 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122fcff-33a1-4e66-b390-674ae920eb7b admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:01.930092 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1122fcff-33a1-4e66-b390-674ae920eb7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b81ac157ef94f598c0ea7d19ac65a45'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:01.948436 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1122fcff-33a1-4e66-b390-674ae920eb7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b81ac157ef94f598c0ea7d19ac65a45'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:01.955991 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1122fcff-33a1-4e66-b390-674ae920eb7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b81ac157ef94f598c0ea7d19ac65a45'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:02.004683 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e16d8-eb0a-461b-af3f-4c52e3f071af service placement] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:02.006217 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-519e16d8-eb0a-461b-af3f-4c52e3f071af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:02.009955 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1122fcff-33a1-4e66-b390-674ae920eb7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b81ac157ef94f598c0ea7d19ac65a45'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:02.013207 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-1122fcff-33a1-4e66-b390-674ae920eb7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b81ac157ef94f598c0ea7d19ac65a45'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:02.020288 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1438/2854] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:03:01 2022] DELETE /identity/v3/users/5b81ac157ef94f598c0ea7d19ac65a45 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:03:02.026184 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108421-b098-4dce-867a-6f8c1f359199 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:02.026615 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108421-b098-4dce-867a-6f8c1f359199 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:02.027027 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108421-b098-4dce-867a-6f8c1f359199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:02.052301 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-64108421-b098-4dce-867a-6f8c1f359199 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:03:02.053598 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-64108421-b098-4dce-867a-6f8c1f359199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:02.092250 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e16d8-eb0a-461b-af3f-4c52e3f071af service placement] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:02.127851 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1439/2855] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 17:03:02.140960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64108421-b098-4dce-867a-6f8c1f359199 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:02.142864 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-64108421-b098-4dce-867a-6f8c1f359199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:02.164840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bc465-3d2f-424c-824d-7021810f5c53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69ee145599384c6da2cba85a42832dcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:02.165523 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bc465-3d2f-424c-824d-7021810f5c53 service nova] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:02.165808 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bc465-3d2f-424c-824d-7021810f5c53 service nova] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:02.166408 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bc465-3d2f-424c-824d-7021810f5c53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:02.167963 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-667bc465-3d2f-424c-824d-7021810f5c53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:02.180735 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bc465-3d2f-424c-824d-7021810f5c53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:02.182360 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bc465-3d2f-424c-824d-7021810f5c53 service nova] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:02.193730 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1417/2856] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:03:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:03:02.205021 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64108421-b098-4dce-867a-6f8c1f359199 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:02.208763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c503f28-3b93-4974-9788-f60dcde152ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:02.209440 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c503f28-3b93-4974-9788-f60dcde152ef service neutron] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:02.209720 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c503f28-3b93-4974-9788-f60dcde152ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:02.210005 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c503f28-3b93-4974-9788-f60dcde152ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:02.220414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c503f28-3b93-4974-9788-f60dcde152ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:02.222083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c503f28-3b93-4974-9788-f60dcde152ef service neutron] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:02.232963 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1440/2857] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:03:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:03:02.749699 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bc465-3d2f-424c-824d-7021810f5c53 service nova] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:02.751127 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-667bc465-3d2f-424c-824d-7021810f5c53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:02.765840 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bc465-3d2f-424c-824d-7021810f5c53 service nova] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:02.771789 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:02.775115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:02.775115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:02.775115 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] PATH_INFO: `/v3/projects/73a4c19197794a55875f263b35f4dd91` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:02.775731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] RBAC: Authorizing `identity:delete_project(project_id=73a4c19197794a55875f263b35f4dd91)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:02.777175 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:02.892126 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73a4c19197794a55875f263b35f4dd91', 'request_id': 'req-49487607-ce36-48d7-b127-7cdaa3b9b45f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:02.899277 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73a4c19197794a55875f263b35f4dd91', 'request_id': 'req-49487607-ce36-48d7-b127-7cdaa3b9b45f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:02.905121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73a4c19197794a55875f263b35f4dd91', 'request_id': 'req-49487607-ce36-48d7-b127-7cdaa3b9b45f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:02.987438 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] The token cache is being invalidate because project 73a4c19197794a55875f263b35f4dd91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80523) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:03:02.987735 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:02.988391 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:02.990269 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1418/2858] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:03:02 2022] DELETE /identity/v3/projects/73a4c19197794a55875f263b35f4dd91 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:03:09.110960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb87dc-f86a-4242-b08c-513dcb8467c9 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:09.111474 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb87dc-f86a-4242-b08c-513dcb8467c9 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:09.111575 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb87dc-f86a-4242-b08c-513dcb8467c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:09.129545 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-1beb87dc-f86a-4242-b08c-513dcb8467c9 None None] MFA Rules not processed for user `6daaa1076f054dd39fdf03017adb0d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:03:09.130764 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1beb87dc-f86a-4242-b08c-513dcb8467c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:09.140057 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:09.186644 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1441/2859] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:03:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:03:09.423370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:09.424793 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:09.436642 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:09.489225 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49487607-ce36-48d7-b127-7cdaa3b9b45f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:09.492824 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaef903-fa22-4e7b-a458-eb7aa611795d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:09.493503 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaef903-fa22-4e7b-a458-eb7aa611795d service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:09.495960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaef903-fa22-4e7b-a458-eb7aa611795d service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:09.495960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaef903-fa22-4e7b-a458-eb7aa611795d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:09.495960 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-faaef903-fa22-4e7b-a458-eb7aa611795d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:09.506375 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaef903-fa22-4e7b-a458-eb7aa611795d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:09.507951 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaef903-fa22-4e7b-a458-eb7aa611795d service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:09.516554 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1419/2860] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:03:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 17:03:19.240293 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=18 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:19.513085 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=23 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:26.595485 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb87dc-f86a-4242-b08c-513dcb8467c9 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:26.596956 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-1beb87dc-f86a-4242-b08c-513dcb8467c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:26.667670 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb87dc-f86a-4242-b08c-513dcb8467c9 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:26.672702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ef640-0648-4ac1-b4d4-2eb598d82f8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:26.673344 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ef640-0648-4ac1-b4d4-2eb598d82f8e service placement] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:26.673649 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ef640-0648-4ac1-b4d4-2eb598d82f8e service placement] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:26.673908 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ef640-0648-4ac1-b4d4-2eb598d82f8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:26.675666 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ff7ef640-0648-4ac1-b4d4-2eb598d82f8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:26.687138 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ef640-0648-4ac1-b4d4-2eb598d82f8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:26.688998 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ef640-0648-4ac1-b4d4-2eb598d82f8e service placement] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:26.698765 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1442/2861] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:03:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:03:32.627982 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6027bd-6225-42d5-a689-a4fb559cbcb5 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:32.628490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6027bd-6225-42d5-a689-a4fb559cbcb5 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:32.628591 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6027bd-6225-42d5-a689-a4fb559cbcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:32.646560 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-ba6027bd-6225-42d5-a689-a4fb559cbcb5 None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:03:32.649567 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-ba6027bd-6225-42d5-a689-a4fb559cbcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:32.706577 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1420/2862] 10.222.0.88 () {58 vars in 1009 bytes} [Mon Jul 18 17:03:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:03:32.836362 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ef640-0648-4ac1-b4d4-2eb598d82f8e service placement] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:32.837793 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-ff7ef640-0648-4ac1-b4d4-2eb598d82f8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:32.921361 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ef640-0648-4ac1-b4d4-2eb598d82f8e service placement] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:32.927106 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df04aa6-d56b-45ab-9f23-c05236ee0622 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:32.927763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04aa6-d56b-45ab-9f23-c05236ee0622 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:32.928050 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04aa6-d56b-45ab-9f23-c05236ee0622 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:32.928329 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df04aa6-d56b-45ab-9f23-c05236ee0622 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:32.937225 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df04aa6-d56b-45ab-9f23-c05236ee0622 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:32.939124 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df04aa6-d56b-45ab-9f23-c05236ee0622 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:32.947561 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1443/2863] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 17:03:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:03:42.700465 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=25 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:42.944210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=69 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:50.340542 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3e38-6635-4492-b751-919c66dc0ebf None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:50.340542 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3e38-6635-4492-b751-919c66dc0ebf None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:50.340542 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3e38-6635-4492-b751-919c66dc0ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:50.358911 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-43bd3e38-6635-4492-b751-919c66dc0ebf None None] MFA Rules not processed for user `3f38abef63e24567a995b30c144177fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:03:50.362490 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-43bd3e38-6635-4492-b751-919c66dc0ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:50.367946 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1421/2864] 10.222.0.88 () {58 vars in 967 bytes} [Mon Jul 18 17:03:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:03:50.380785 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df04aa6-d56b-45ab-9f23-c05236ee0622 service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:50.392404 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df04aa6-d56b-45ab-9f23-c05236ee0622 service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:50.398594 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f27ec0-bc93-48f5-bf7b-af6d3e995662 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf341f1bb8944a3e9331678100b941a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:50.399250 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f27ec0-bc93-48f5-bf7b-af6d3e995662 service swift] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:50.399537 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f27ec0-bc93-48f5-bf7b-af6d3e995662 service swift] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:50.399847 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f27ec0-bc93-48f5-bf7b-af6d3e995662 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:50.408709 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f27ec0-bc93-48f5-bf7b-af6d3e995662 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:50.410890 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f27ec0-bc93-48f5-bf7b-af6d3e995662 service swift] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:50.419423 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1444/2865] 10.222.0.88 () {60 vars in 1384 bytes} [Mon Jul 18 17:03:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 17:03:53.426964 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd3e38-6635-4492-b751-919c66dc0ebf None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:53.429726 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-43bd3e38-6635-4492-b751-919c66dc0ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:53.511877 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd3e38-6635-4492-b751-919c66dc0ebf None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:53.516843 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc12b8-f14a-4cd7-8d46-6dd96e3fc237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:53.517869 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc12b8-f14a-4cd7-8d46-6dd96e3fc237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:53.518276 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc12b8-f14a-4cd7-8d46-6dd96e3fc237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:53.518670 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc12b8-f14a-4cd7-8d46-6dd96e3fc237 admin admin] PATH_INFO: `/v3/users/6daaa1076f054dd39fdf03017adb0d2f` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:53.520686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc12b8-f14a-4cd7-8d46-6dd96e3fc237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6daaa1076f054dd39fdf03017adb0d2f)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:53.522635 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc12b8-f14a-4cd7-8d46-6dd96e3fc237 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:53.744678 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-91fc12b8-f14a-4cd7-8d46-6dd96e3fc237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6daaa1076f054dd39fdf03017adb0d2f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:53.755574 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-91fc12b8-f14a-4cd7-8d46-6dd96e3fc237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6daaa1076f054dd39fdf03017adb0d2f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:53.772616 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-91fc12b8-f14a-4cd7-8d46-6dd96e3fc237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6daaa1076f054dd39fdf03017adb0d2f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:53.861956 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-91fc12b8-f14a-4cd7-8d46-6dd96e3fc237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6daaa1076f054dd39fdf03017adb0d2f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:53.862598 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-91fc12b8-f14a-4cd7-8d46-6dd96e3fc237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6daaa1076f054dd39fdf03017adb0d2f'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:53.869453 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1422/2866] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:03:53 2022] DELETE /identity/v3/users/6daaa1076f054dd39fdf03017adb0d2f => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:03:54.125929 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f27ec0-bc93-48f5-bf7b-af6d3e995662 service swift] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:54.138573 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f27ec0-bc93-48f5-bf7b-af6d3e995662 service swift] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:54.201214 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:54.202094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:54.202382 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:54.202683 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] PATH_INFO: `/v3/projects/18f27a3568f447279e227939696730bf` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:54.203945 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=18f27a3568f447279e227939696730bf)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:54.205935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:54.320796 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18f27a3568f447279e227939696730bf', 'request_id': 'req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:54.402657 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18f27a3568f447279e227939696730bf', 'request_id': 'req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:54.408856 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18f27a3568f447279e227939696730bf', 'request_id': 'req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:54.423281 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] The token cache is being invalidate because project 18f27a3568f447279e227939696730bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:03:54.423564 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:54.424210 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-00da8298-c9d8-4910-b9a1-3f94eaa8eecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:03:54.425827 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1445/2867] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:03:54 2022] DELETE /identity/v3/projects/18f27a3568f447279e227939696730bf => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:03:54.431653 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fc36b-45ab-4f2a-b9c7-d77dcddf54e4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:54.432188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fc36b-45ab-4f2a-b9c7-d77dcddf54e4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:54.432188 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fc36b-45ab-4f2a-b9c7-d77dcddf54e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:54.450007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-a05fc36b-45ab-4f2a-b9c7-d77dcddf54e4 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:03:54.451219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a05fc36b-45ab-4f2a-b9c7-d77dcddf54e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:54.511608 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1423/2868] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:03:54.519191 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ed1f0-a235-48cf-9258-72723ae785f8 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:54.519473 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ed1f0-a235-48cf-9258-72723ae785f8 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:54.519744 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ed1f0-a235-48cf-9258-72723ae785f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:54.539212 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-412ed1f0-a235-48cf-9258-72723ae785f8 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:03:54.540418 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-412ed1f0-a235-48cf-9258-72723ae785f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:54.548381 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1446/2869] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:03:54.553035 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fc36b-45ab-4f2a-b9c7-d77dcddf54e4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:54.565532 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fc36b-45ab-4f2a-b9c7-d77dcddf54e4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:54.570888 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab769ae-53d9-447a-9cc8-c72e0b37dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:54.571575 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab769ae-53d9-447a-9cc8-c72e0b37dcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:54.571835 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab769ae-53d9-447a-9cc8-c72e0b37dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:54.572117 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab769ae-53d9-447a-9cc8-c72e0b37dcf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:54.572916 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab769ae-53d9-447a-9cc8-c72e0b37dcf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:03:54.573214 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab769ae-53d9-447a-9cc8-c72e0b37dcf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:54.574880 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab769ae-53d9-447a-9cc8-c72e0b37dcf2 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:54.584482 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:54.587913 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1424/2870] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:03:54.592465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ed1f0-a235-48cf-9258-72723ae785f8 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:54.601790 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ed1f0-a235-48cf-9258-72723ae785f8 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:54.606558 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06753f1f-4631-4434-980e-974a81312669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:54.607205 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06753f1f-4631-4434-980e-974a81312669 admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:54.607488 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06753f1f-4631-4434-980e-974a81312669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:54.607792 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06753f1f-4631-4434-980e-974a81312669 admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:54.608494 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06753f1f-4631-4434-980e-974a81312669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:54.610425 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06753f1f-4631-4434-980e-974a81312669 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:54.732087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1447/2871] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:03:54 2022] POST /identity/v3/projects => generated 367 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:03:54.736287 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab769ae-53d9-447a-9cc8-c72e0b37dcf2 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:54.747510 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab769ae-53d9-447a-9cc8-c72e0b37dcf2 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:54.805042 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f02a49-5b27-46cc-b775-0c6a378a34ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:54.805714 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f02a49-5b27-46cc-b775-0c6a378a34ed admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:54.806001 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f02a49-5b27-46cc-b775-0c6a378a34ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:54.806281 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f02a49-5b27-46cc-b775-0c6a378a34ed admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:54.806971 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f02a49-5b27-46cc-b775-0c6a378a34ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:54.808910 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f02a49-5b27-46cc-b775-0c6a378a34ed admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:54.917442 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1425/2872] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:03:54 2022] POST /identity/v3/users => generated 284 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:03:54.923317 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06753f1f-4631-4434-980e-974a81312669 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:54.934982 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06753f1f-4631-4434-980e-974a81312669 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:54.940016 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86c7ad-9aa0-42ae-937a-1dffdd5911ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:54.940687 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c7ad-9aa0-42ae-937a-1dffdd5911ca admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:54.941007 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c7ad-9aa0-42ae-937a-1dffdd5911ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:54.941286 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c7ad-9aa0-42ae-937a-1dffdd5911ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:54.942637 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86c7ad-9aa0-42ae-937a-1dffdd5911ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:54.944272 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86c7ad-9aa0-42ae-937a-1dffdd5911ca admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:54.955161 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1448/2873] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:03:54.959207 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f02a49-5b27-46cc-b775-0c6a378a34ed admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:54.969376 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f02a49-5b27-46cc-b775-0c6a378a34ed admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:54.974053 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52094dcc-170d-41d9-89cd-8cc072ac3bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:54.974704 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52094dcc-170d-41d9-89cd-8cc072ac3bed admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:54.974990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52094dcc-170d-41d9-89cd-8cc072ac3bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:54.975267 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52094dcc-170d-41d9-89cd-8cc072ac3bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:54.975991 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52094dcc-170d-41d9-89cd-8cc072ac3bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:54.977636 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52094dcc-170d-41d9-89cd-8cc072ac3bed admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:54.987850 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1426/2874] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:03:54.992351 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86c7ad-9aa0-42ae-937a-1dffdd5911ca admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:55.002398 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86c7ad-9aa0-42ae-937a-1dffdd5911ca admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:55.007663 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b34310-15cc-4548-88d3-cefaf24721b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:55.008540 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b34310-15cc-4548-88d3-cefaf24721b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:55.008825 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b34310-15cc-4548-88d3-cefaf24721b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:55.009135 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b34310-15cc-4548-88d3-cefaf24721b8 admin admin] PATH_INFO: `/v3/projects/d9a587f63e614fcb95bf994c896879a8/users/01094127cacc43ac9bfc0bf32dddfd40/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:55.023702 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b34310-15cc-4548-88d3-cefaf24721b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a587f63e614fcb95bf994c896879a8, user_id=01094127cacc43ac9bfc0bf32dddfd40, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:55.026400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b34310-15cc-4548-88d3-cefaf24721b8 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:55.131023 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1449/2875] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:03:54 2022] PUT /identity/v3/projects/d9a587f63e614fcb95bf994c896879a8/users/01094127cacc43ac9bfc0bf32dddfd40/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:03:55.141689 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacbfbf-4c92-40f1-ab24-0687741386a4 None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:55.141979 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacbfbf-4c92-40f1-ab24-0687741386a4 None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:55.142257 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacbfbf-4c92-40f1-ab24-0687741386a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:55.168141 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-deacbfbf-4c92-40f1-ab24-0687741386a4 None None] MFA Rules not processed for user `01094127cacc43ac9bfc0bf32dddfd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:03:55.169432 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-deacbfbf-4c92-40f1-ab24-0687741386a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:55.234018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1427/2876] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:03:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:03:55.245426 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b34310-15cc-4548-88d3-cefaf24721b8 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:03:55.247136 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-29b34310-15cc-4548-88d3-cefaf24721b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:03:55.266047 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:03:55.330393 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b34310-15cc-4548-88d3-cefaf24721b8 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:03:55.334877 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842d789-ee24-4f15-9369-ac27dad51f3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54c7869983bf4534b3e730af0185eac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:03:55.340178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842d789-ee24-4f15-9369-ac27dad51f3d service glance] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:03:55.340178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842d789-ee24-4f15-9369-ac27dad51f3d service glance] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:03:55.340178 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842d789-ee24-4f15-9369-ac27dad51f3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:03:55.345271 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842d789-ee24-4f15-9369-ac27dad51f3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:03:55.346705 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842d789-ee24-4f15-9369-ac27dad51f3d service glance] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:03:55.355596 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1450/2877] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:03:55 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:04:05.231213 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:04:05.358609 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=29 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:04:28.231553 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacbfbf-4c92-40f1-ab24-0687741386a4 None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:28.294372 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacbfbf-4c92-40f1-ab24-0687741386a4 None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:28.299011 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:28.299872 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:28.300160 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:28.300437 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] PATH_INFO: `/v3/users/01094127cacc43ac9bfc0bf32dddfd40` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:28.301656 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01094127cacc43ac9bfc0bf32dddfd40)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:28.303646 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:28.495739 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01094127cacc43ac9bfc0bf32dddfd40'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:28.502662 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01094127cacc43ac9bfc0bf32dddfd40'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:28.514849 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01094127cacc43ac9bfc0bf32dddfd40'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:28.574845 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01094127cacc43ac9bfc0bf32dddfd40'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:28.575480 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01094127cacc43ac9bfc0bf32dddfd40'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:28.581900 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1428/2878] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:04:28 2022] DELETE /identity/v3/users/01094127cacc43ac9bfc0bf32dddfd40 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:04:28.586885 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0cdfa-ed3c-427e-9b48-214f511a71c4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:28.589082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0cdfa-ed3c-427e-9b48-214f511a71c4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:28.589082 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0cdfa-ed3c-427e-9b48-214f511a71c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:28.605085 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-c4f0cdfa-ed3c-427e-9b48-214f511a71c4 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:04:28.607040 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c4f0cdfa-ed3c-427e-9b48-214f511a71c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:28.666933 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1451/2879] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:04:28.677733 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:28.678416 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:28.740557 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152e63f-5c58-4557-85cb-526643d30414 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:28.744351 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7424e03-2aad-44af-88ba-cfe8963727b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:28.745048 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7424e03-2aad-44af-88ba-cfe8963727b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:28.745382 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7424e03-2aad-44af-88ba-cfe8963727b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:28.745621 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7424e03-2aad-44af-88ba-cfe8963727b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:28.755018 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7424e03-2aad-44af-88ba-cfe8963727b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:28.758994 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7424e03-2aad-44af-88ba-cfe8963727b1 service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:28.770973 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1429/2880] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:04:29.176471 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0cdfa-ed3c-427e-9b48-214f511a71c4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:29.179848 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-c4f0cdfa-ed3c-427e-9b48-214f511a71c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:29.195213 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0cdfa-ed3c-427e-9b48-214f511a71c4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:29.199942 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:29.200811 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:29.201119 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:29.201392 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] PATH_INFO: `/v3/projects/d9a587f63e614fcb95bf994c896879a8` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:29.202677 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a587f63e614fcb95bf994c896879a8)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:29.204515 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:29.307766 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a587f63e614fcb95bf994c896879a8', 'request_id': 'req-9b405f4c-5eb8-46cc-9121-63ffda49f458'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:29.389734 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a587f63e614fcb95bf994c896879a8', 'request_id': 'req-9b405f4c-5eb8-46cc-9121-63ffda49f458'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:29.396063 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a587f63e614fcb95bf994c896879a8', 'request_id': 'req-9b405f4c-5eb8-46cc-9121-63ffda49f458'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:29.408902 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] The token cache is being invalidate because project d9a587f63e614fcb95bf994c896879a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:04:29.410238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:29.410238 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-9b405f4c-5eb8-46cc-9121-63ffda49f458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:29.411590 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1452/2881] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:04:29 2022] DELETE /identity/v3/projects/d9a587f63e614fcb95bf994c896879a8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:04:29.416824 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4282e06-57eb-4e3f-9941-c6d8b3fb73da None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:29.417345 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4282e06-57eb-4e3f-9941-c6d8b3fb73da None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:29.417463 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4282e06-57eb-4e3f-9941-c6d8b3fb73da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:29.435675 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-b4282e06-57eb-4e3f-9941-c6d8b3fb73da None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:04:29.437222 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-b4282e06-57eb-4e3f-9941-c6d8b3fb73da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:29.455121 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:04:29.500032 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1430/2882] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:04:29.508509 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d5be-dcf4-43e9-bbd3-64abecd8e041 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:29.508805 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d5be-dcf4-43e9-bbd3-64abecd8e041 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:29.509134 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d5be-dcf4-43e9-bbd3-64abecd8e041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:29.528279 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-4953d5be-dcf4-43e9-bbd3-64abecd8e041 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:04:29.531452 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-4953d5be-dcf4-43e9-bbd3-64abecd8e041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:29.537638 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1453/2883] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:04:29.541909 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4282e06-57eb-4e3f-9941-c6d8b3fb73da None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:29.552013 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4282e06-57eb-4e3f-9941-c6d8b3fb73da None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:29.556860 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bce4d-aeec-4cdd-923c-84b5e372db3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:29.557536 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bce4d-aeec-4cdd-923c-84b5e372db3d admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:29.557874 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bce4d-aeec-4cdd-923c-84b5e372db3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:29.558140 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bce4d-aeec-4cdd-923c-84b5e372db3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:29.558830 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bce4d-aeec-4cdd-923c-84b5e372db3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80523) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 17:04:29.559134 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bce4d-aeec-4cdd-923c-84b5e372db3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:29.560731 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666bce4d-aeec-4cdd-923c-84b5e372db3d admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:29.573113 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1431/2884] 10.222.0.88 () {58 vars in 1152 bytes} [Mon Jul 18 17:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 17:04:29.577465 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953d5be-dcf4-43e9-bbd3-64abecd8e041 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:29.587804 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953d5be-dcf4-43e9-bbd3-64abecd8e041 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:29.592838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7b40e-a758-4296-bd2e-b057598ff0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:29.593528 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b40e-a758-4296-bd2e-b057598ff0ff admin admin] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:29.593822 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b40e-a758-4296-bd2e-b057598ff0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:29.594128 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b40e-a758-4296-bd2e-b057598ff0ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:29.594859 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7b40e-a758-4296-bd2e-b057598ff0ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:29.596774 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7b40e-a758-4296-bd2e-b057598ff0ff admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:29.682779 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1454/2885] 10.222.0.88 () {60 vars in 1152 bytes} [Mon Jul 18 17:04:29 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:04:29.687157 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bce4d-aeec-4cdd-923c-84b5e372db3d admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:29.697927 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bce4d-aeec-4cdd-923c-84b5e372db3d admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:29.757686 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d1a9a-dfdd-4665-8529-426e6a4a5408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:29.758370 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1a9a-dfdd-4665-8529-426e6a4a5408 admin admin] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:29.758677 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1a9a-dfdd-4665-8529-426e6a4a5408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:29.758948 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1a9a-dfdd-4665-8529-426e6a4a5408 admin admin] PATH_INFO: `/v3/users` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:29.759633 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d1a9a-dfdd-4665-8529-426e6a4a5408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:29.761578 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d1a9a-dfdd-4665-8529-426e6a4a5408 admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:29.870895 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1432/2886] 10.222.0.88 () {60 vars in 1143 bytes} [Mon Jul 18 17:04:29 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 17:04:29.875059 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7b40e-a758-4296-bd2e-b057598ff0ff admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:29.885621 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7b40e-a758-4296-bd2e-b057598ff0ff admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:29.890619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87178b-81ce-4526-a6a6-7f76448b606a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:29.891308 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87178b-81ce-4526-a6a6-7f76448b606a admin admin] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:29.891593 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87178b-81ce-4526-a6a6-7f76448b606a admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:29.891872 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87178b-81ce-4526-a6a6-7f76448b606a admin admin] PATH_INFO: `/v3/roles` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:29.892606 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87178b-81ce-4526-a6a6-7f76448b606a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:29.894277 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87178b-81ce-4526-a6a6-7f76448b606a admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:29.906934 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1455/2887] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:04:29.910989 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d1a9a-dfdd-4665-8529-426e6a4a5408 admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:29.922088 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d1a9a-dfdd-4665-8529-426e6a4a5408 admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:29.926975 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ca963-bd2c-41eb-ab19-6ffd0b1c817b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:29.927643 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ca963-bd2c-41eb-ab19-6ffd0b1c817b admin admin] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:29.927955 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ca963-bd2c-41eb-ab19-6ffd0b1c817b admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:29.928224 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ca963-bd2c-41eb-ab19-6ffd0b1c817b admin admin] PATH_INFO: `/v3/roles` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:29.928990 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ca963-bd2c-41eb-ab19-6ffd0b1c817b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:29.930695 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ca963-bd2c-41eb-ab19-6ffd0b1c817b admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:29.942205 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1433/2888] 10.222.0.88 () {58 vars in 1121 bytes} [Mon Jul 18 17:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 17:04:29.946338 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87178b-81ce-4526-a6a6-7f76448b606a admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:29.956763 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87178b-81ce-4526-a6a6-7f76448b606a admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:29.961960 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b030b7c-cfe4-4434-938b-5b3957e23050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:29.962844 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b030b7c-cfe4-4434-938b-5b3957e23050 admin admin] REQUEST_METHOD: `PUT` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:29.963132 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b030b7c-cfe4-4434-938b-5b3957e23050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:29.963410 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b030b7c-cfe4-4434-938b-5b3957e23050 admin admin] PATH_INFO: `/v3/projects/9c980bfed4294fe7a287633274da3efc/users/5ac0990db28a4680b8722af435d0eacc/roles/3b2e1fad42f441e9a61c142477a50b11` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:29.978838 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b030b7c-cfe4-4434-938b-5b3957e23050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c980bfed4294fe7a287633274da3efc, user_id=5ac0990db28a4680b8722af435d0eacc, role_id=3b2e1fad42f441e9a61c142477a50b11)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:29.981544 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b030b7c-cfe4-4434-938b-5b3957e23050 admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:30.050397 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1456/2889] 10.222.0.88 () {60 vars in 1482 bytes} [Mon Jul 18 17:04:29 2022] PUT /identity/v3/projects/9c980bfed4294fe7a287633274da3efc/users/5ac0990db28a4680b8722af435d0eacc/roles/3b2e1fad42f441e9a61c142477a50b11 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:04:30.061919 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbf5a5-98bb-4e25-a95c-2c4e67ae67fa None None] REQUEST_METHOD: `POST` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:30.062212 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbf5a5-98bb-4e25-a95c-2c4e67ae67fa None None] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:30.062493 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbf5a5-98bb-4e25-a95c-2c4e67ae67fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:30.091498 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.auth.core [None req-97dbf5a5-98bb-4e25-a95c-2c4e67ae67fa None None] MFA Rules not processed for user `5ac0990db28a4680b8722af435d0eacc`. Rule list: `[]` (Enabled: `True`). {{(pid=80523) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:04:30.092870 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-97dbf5a5-98bb-4e25-a95c-2c4e67ae67fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:30.161219 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1434/2890] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:04:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:04:30.172511 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b030b7c-cfe4-4434-938b-5b3957e23050 admin admin] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:30.174306 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-5b030b7c-cfe4-4434-938b-5b3957e23050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:30.196846 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:04:30.232094 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b030b7c-cfe4-4434-938b-5b3957e23050 admin admin] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:30.235773 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cafc3b-2dd9-498e-8944-35761138572b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '809604baaed7486fb1020263e99e9c63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:30.236414 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cafc3b-2dd9-498e-8944-35761138572b service cinder] REQUEST_METHOD: `GET` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:30.236706 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cafc3b-2dd9-498e-8944-35761138572b service cinder] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:30.237181 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cafc3b-2dd9-498e-8944-35761138572b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:30.245500 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cafc3b-2dd9-498e-8944-35761138572b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:30.246935 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cafc3b-2dd9-498e-8944-35761138572b service cinder] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:30.255636 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1457/2891] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:04:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:04:33.344622 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbf5a5-98bb-4e25-a95c-2c4e67ae67fa None None] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:33.363394 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:04:33.406727 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbf5a5-98bb-4e25-a95c-2c4e67ae67fa None None] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:33.411307 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:33.412206 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:33.412486 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:33.412791 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] PATH_INFO: `/v3/users/5ac0990db28a4680b8722af435d0eacc` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:33.414007 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ac0990db28a4680b8722af435d0eacc)` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:33.415998 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:33.623846 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ac0990db28a4680b8722af435d0eacc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:33.630752 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ac0990db28a4680b8722af435d0eacc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:33.643603 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ac0990db28a4680b8722af435d0eacc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:33.688859 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ac0990db28a4680b8722af435d0eacc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:33.689531 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.notifications [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ac0990db28a4680b8722af435d0eacc'} {{(pid=80523) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:33.696451 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1435/2892] 10.222.0.88 () {58 vars in 1223 bytes} [Mon Jul 18 17:04:33 2022] DELETE /identity/v3/users/5ac0990db28a4680b8722af435d0eacc => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:04:33.701083 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee43117-1861-424d-a04f-ad946418f4c4 None None] REQUEST_METHOD: `POST` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:33.701599 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee43117-1861-424d-a04f-ad946418f4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:33.701716 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee43117-1861-424d-a04f-ad946418f4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:33.718959 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.auth.core [None req-dee43117-1861-424d-a04f-ad946418f4c4 None None] MFA Rules not processed for user `264c9bf15b9849a18b942d5d77559b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 17:04:33.720159 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dee43117-1861-424d-a04f-ad946418f4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:33.778219 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1458/2893] 10.222.0.88 () {58 vars in 957 bytes} [Mon Jul 18 17:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 17:04:33.787861 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:33.789440 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:33.846930 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5825ea-0bee-4903-8b3d-1bf12412d03f admin admin] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:33.851141 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813b2cb-4a82-4a5b-97e4-7adb15127c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de8484c3b114467ea59bebb2460ed546', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:33.851783 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813b2cb-4a82-4a5b-97e4-7adb15127c2b service neutron] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:33.852068 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813b2cb-4a82-4a5b-97e4-7adb15127c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:33.852347 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813b2cb-4a82-4a5b-97e4-7adb15127c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:33.861518 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813b2cb-4a82-4a5b-97e4-7adb15127c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:33.862944 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813b2cb-4a82-4a5b-97e4-7adb15127c2b service neutron] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:33.873241 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1436/2894] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:04:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:04:34.277339 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee43117-1861-424d-a04f-ad946418f4c4 None None] Authenticating user token {{(pid=80522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:04:34.279087 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.fernet_utils [None req-dee43117-1861-424d-a04f-ad946418f4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:04:34.293510 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee43117-1861-424d-a04f-ad946418f4c4 None None] Validating token access rules against request {{(pid=80522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:04:34.299221 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '264c9bf15b9849a18b942d5d77559b9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fb46b30ed74742b9a95d61944ef39381', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:04:34.300676 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:04:34.300815 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] SCRIPT_NAME: `/identity` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:04:34.301160 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] PATH_INFO: `/v3/projects/9c980bfed4294fe7a287633274da3efc` {{(pid=80522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:04:34.302400 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c980bfed4294fe7a287633274da3efc)` {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:04:34.304487 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] RBAC: Authorization granted {{(pid=80522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:04:34.408028 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c980bfed4294fe7a287633274da3efc', 'request_id': 'req-bd073237-1f43-493b-b431-f3a97a5552de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:34.501046 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c980bfed4294fe7a287633274da3efc', 'request_id': 'req-bd073237-1f43-493b-b431-f3a97a5552de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:34.507195 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c980bfed4294fe7a287633274da3efc', 'request_id': 'req-bd073237-1f43-493b-b431-f3a97a5552de'} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:34.518332 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] The token cache is being invalidate because project 9c980bfed4294fe7a287633274da3efc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 17:04:34.518619 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:34.519275 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG keystone.notifications [None req-bd073237-1f43-493b-b431-f3a97a5552de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 17:04:34.520839 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: [pid: 80522|app: 0|req: 1459/2895] 10.222.0.88 () {58 vars in 1232 bytes} [Mon Jul 18 17:04:34 2022] DELETE /identity/v3/projects/9c980bfed4294fe7a287633274da3efc => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 17:04:43.868904 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:04:44.515333 dv-cinder-846173-3-iscsi devstack@keystone.service[80522]: DEBUG dbcounter [-] [80522] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 17:09:07.157665 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813b2cb-4a82-4a5b-97e4-7adb15127c2b service neutron] Authenticating user token {{(pid=80523) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 17:09:07.159076 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-a813b2cb-4a82-4a5b-97e4-7adb15127c2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:09:07.352920 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813b2cb-4a82-4a5b-97e4-7adb15127c2b service neutron] Validating token access rules against request {{(pid=80523) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 17:09:07.413364 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a471ee7-f403-4373-bc1e-c72bfb8c44bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22bd4dab1fcd4ad4a8be886646d89242', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ec2d156472e4d6a852c592e579f2f39', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80523) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 17:09:07.414030 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471ee7-f403-4373-bc1e-c72bfb8c44bc service placement] REQUEST_METHOD: `GET` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 17:09:07.414311 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471ee7-f403-4373-bc1e-c72bfb8c44bc service placement] SCRIPT_NAME: `/identity` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 17:09:07.414590 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471ee7-f403-4373-bc1e-c72bfb8c44bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80523) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 17:09:07.416506 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.fernet_utils [None req-4a471ee7-f403-4373-bc1e-c72bfb8c44bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80523) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 17:09:07.429010 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a471ee7-f403-4373-bc1e-c72bfb8c44bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 17:09:07.430881 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a471ee7-f403-4373-bc1e-c72bfb8c44bc service placement] RBAC: Authorization granted {{(pid=80523) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 17:09:07.442799 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: [pid: 80523|app: 0|req: 1437/2896] 10.222.0.88 () {60 vars in 1365 bytes} [Mon Jul 18 17:09:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 17:09:17.437176 dv-cinder-846173-3-iscsi devstack@keystone.service[80523]: DEBUG dbcounter [-] [80523] Writing DB stats keystone:SELECT=89 {{(pid=80523) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}